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Preface 
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Abstract 


Thisste^  investigated  differences  in  organizational  efficiency  and 
effectiveness  for  users  of  graphical  user  interfaces  and  text-based  interfaces 
on  personal  computei  in  Air  Force  offices.  Areas  of  interest  included 
amount  of  time  required  to  learn  the  basic  system,  amount  of  time  required 
to  learn  new  applications,  users’  ratings  of  user-friendliness,  users’ 
perceptions  of  the  extent  that  their  system  helps  them  perform  in  their  job, 
the  number  of  software  packages  used  on  the  job  by  users  of  each  system, 
user  satisfaction,  responsible  authorities’  ratings  of  quality  of  output,  and 
the  relationship  between  user  job  experience  level  and  interface  used  on  the 
job.  A  literature  review  revealed  no  similar  studies  within  the  Department 
of  Defense  to  date.  Two  populations  across  two  organizations  were 
identified  for  survey  administration-users  of  graphical  user  interfaces  and 
users  of  text-based  interfaces.  A  total  of  700  surveys  were  sent  out  with 
454  returned  for  a  response  rate  of  64.9%.  The  results  of  the  study 
indicated  that  for  the  organizations  surveyed,  graphical  user  interfaces  offer 
significant  advantages  in  each  of  the  areas  investigated.  Finally,  the  results 
of  the  study  revealed  that  less  experienced  users  tend  to  use  graphical 
interfaces  over  text-based  interfaces .  In  addition,  military  members  used 


graphics-based  systems  in  greater  numbers  while  civilian Ajsers  were  more 

-  y  ;  ^  .!  '  ^ 

likely  to  use  text-based  systems.  _ 
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A  COMPARISON  OF  HUMAN-COMPUTER  INTERFACES  IN 
AIR  FORCE  ORGANIZATIONS 


I.  Introduction 

This  chapter  contains  a  general  background  on  human-computer 
interfaces  and  their  use  in  Air  Force  organizations.  The  chapter  provides 
information  on  the  increasing  popularity  of  graphical  user  interfaces  and 
their  impact  on  measures  of  job  efficiency  and  effectiveness.  The  specific 
purpose  of  the  research  is  stated,  and  the  specific  research  questions  are 
listed.  Also  included  are  limitations  to  the  interface  comparison. 

BacKgrQund 

In  Management  Information  Systems:  Conceptual  Foundations. 
Structure,  and  Development.  Gordon  B.  Davis  comments  on  the  role  of  the 
interface  in  any  management  information  system  (MIS).  He  states  that  the 
emphasis  in  information  systems  has  shifted  away  from  machine  efficiency 
to  human  effectiveness.  His  explanation  for  this  is  economic— hardware 
costs  have  declined  relative  to  labor  costs  (6:333).  Therefore.  Davis  asserts 
that  it  is  incumbent  upon  management  to  maximize  user  productivity. 

Davis  contends  that  if  the  user  controls  the  interaction,  he  is  likely  to 
increase  his  level  of  performance.  Normally  this  increased  performance 
manifests  itself  through  lower  turnover  and  increased  job  satisfaction. 
Davis  concludes  that  well-designed  interfaces  lead  to  impressive 
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performance,  and  therefore,  should  be  of  primary  concern  to  managers  and 
system  designers  (6:533). 

As  might  be  predicted  from  the  ideas  expressed  by  Davis  (6).  as  well  as 
those  of  Schneiderman  (25,  26).  Carey  (5).  and  Norman  (23).  graphical  user 
interfaces  are  increasing  in  popularity.  Virtually  all  major  microcomputer 
operating  systems  have  implemented,  or  are  developing,  some  form  of 
graphical  user  interface.  The  growing  acceptance  of  the  Macintosh 
computer  in  the  business  community,  as  well  as  IBM’s  development  of  their 
Presentation  Manager  interface  suggests  that  graphics-based  systems  are 
likely  to  remain  a  focus  for  systems  designers  well  into  the  next  decade. 

Research  on  the  effect  graphical  user  interfaces  have  on  job 
performance  is  contradictory.  Ives.  Olson,  and  Baroudi  have  correlated 
successful  information  systems  implementation  with  user  satisfaction  with 
that  system  (18:785-787).  Their  results  have  challenged  information 
managers  to  successfully  meet  the  needs  of  computer  users  within  the 
organization.  To  that  end.  Diagnostic  Research,  an  independent  research 
corporation,  has  shown  that  a  system  utilizing  a  graphical  user  interface  has 
higher  ratings  from  both  managers  and  users  on  user-friendliness.  In 
addition,  the  graphics-based  system  is  easier  to  lea-n  and  creates  greater 
feelings  of  confidence  on  the  part  of  users  of  that  system  than  users  of  a 
major  character-based  system  (19:10).  Gittins  has  reported  that  graphical 
systems  which  use  icons  can  increase  computing  accuracy  because  icons 
reduce  user  reliance  on  memory  for  valid  command  sets  (9:526).  These 
studies  suggest  that  the  use  of  graphical  interface  systems  has  potential  for 
increasing  user  satisfaction,  reducing  training  time,  and  improving 
accuracy. 
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However,  other  studies  have  contradicted  the  notion  that  the  user 
interface  has  an  important  impact  on  job  efficiency  and  effectiveness.  In 
1985,  Whiteside  compared  user  performance  on  command,  menu,  and  iconic 
interfaces.  His  results  suggested  that  iconic  interfaces  do  not  enhance 
performance  and  that  command-driven  systems  provide  just  as  much 
accuracy  as  other  types  of  systems.  Additionally,  Whiteside  found  no 
evidence  of  user  preference  for  any  of  the  three  alternatives  studied 
(27:190).  Other  academicians  have  criticized  the  graphical  systems’ 
reliance  on  the  real  world  to  simulate  objects.  This  group  of  authors, 
including  Grudin  and  Johnson,  feel  that  simulating  real  world  objects 
hinders  technological  progress  and  fails  to  fully  address  the  advantages 
offered  by  automation  (10:1 166,  16:22).  These  results  cast  doubt  about  the 
advantages  offered  by  graphical  interface  systems  and  suggest  a  need  for 
further  research  in  the  area  to  better  determine  the  role  the  human- 
computer  interface  plays  in  improving  user  performance. 

Most  of  the  microcomputers  in  use  in  the  Air  Force  today  use  the  MS- 
DOS  operating  system-a  character-based,  command-driven  system. 
However,  no  studies  have  been  conducted  within  the  Air  Force  to  compare 
the  standard  MS-DOS  operating  systems  to  alternate  computer  systems 
using  graphical  interfaces.  By  comparing  the  two  systems,  one  can 
determine  if  any  differences  exist  between  character  based  and  graphics- 
based  systems,  and  if  differences  do  exist,  what  the  nature  of  these 
differences  are,  and  what  implications  these  results  have  for  Air  Force 
information  managers. 
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Problem  Statement 


As  a  result  of  the  debate  about  the  effectiveness  of  graphics  in  computer 
operating  systems  and  the  growing  popularity  of  such  systems  in  the 
civilian  community,  the  researcher  determined  that  there  was  a  need  for 
further  research  into  the  effect  graphical  user  interfaces  have  in  an  Air 
Force  environment.  The  following  research  question  best  identifies  the 
management  issue  described  above: 

What  are  the  advantages  and  disadvantages  of  graphical,  mouse-driven 
user  interfaces  as  compared  to  character-based,  command-driven  user 
interfaces,  and  what  is  each  system’s  impact  on  measures  of  job  efficiency 
and  job  effectiveness  within  the  Air  Force? 

Investigative  Questions 

To  best  answer  the  research  question,  the  following  investigative 
questions  must  be  answered; 

1 .  How  does  each  type  of  interface  compare  in  the  amount  of  time 
required  to  learn  the  basic  system? 

2.  How  does  each  type  of  interface  compare  in  the  amount  of  time  users 
report  is  typically  required  to  learn  a  new  application  program? 

3.  How  does  each  type  of  interface  compare  in  users’  ratings  of  user- 
friendliness? 

4.  How  does  each  type  of  interface  affect  users’  perceptions  of  the 
extent  that  their  computer  system  helps  them  perform  in  their  job? 

5.  How  does  each  type  of  interface  affect  the  number  of  software 
packages  users  of  that  system  use  on  the  job? 

6.  How  does  each  type  of  interface  affect  user  satisfaction  with  their 
computer  system? 


4 


7.  How  does  each  type  of  system  compare  in  responsible  authorities’ 
ratings  of  quality  of  output? 

8.  What  is  the  nature  of  the  relationship  between  users’  job  experience 
level  and  type  of  interface  used  on  the  job? 

Limitations 

This  study  compared  users  of  the  Apple  Macintosh  and  the  Zenith  Z-248 
computer  systems.  The  Macintosh  was  chosen  because  it  is  the  oldest  and 
most  widely  used  of  all  graphical  user  interface  systems  within  the  Air 
Force.  Likewise,  the  Zenith  Z-248  is  the  standard  character- based  system 
in  use  within  the  Air  Force  today. 

In  addition,  it  is  possible  to  examine  organizations  with  similar  missions 
that  use  both  MS-DOS  systems  and  Macintoshes.  This  is  not  possible  with 
other  graphical  interface  systems  at  the  present  time.  For  this  reason,  the 
researcher  chose  to  limit  his  study  to  organizations  within  the  Aeronautical 
Systems  Division,  Air  Force  Systems  Command  located  at  Wright-Patterson 
Air  Force  Base,  Ohio.  By  focusing  on  similar  organizations  within  the  same 
command,  the  researcher  was  able  to  account  for  potential  organizational 
differences  in  making  inferences  based  on  the  results  of  the  research. 

Finally,  to  assure  that  computer  users  responding  to  the  survey  were 
knowledgeable,  the  researcher  only  considered  responses  from  computer 
users  who  reported  that  they  use  their  computer  at  least  three  hours  per 
week,  have  used  the  particular  operating  system  for  three  months  or  more, 
and  are  familiar  with  more  than  one  software  application. 
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II.  Literature  Review 


This  chapter  provides  a  review  of  significant  articles  and  research 
contributions  in  the  field.  Early  models  for  interface  design,  cognitive 
issues,  the  use  of  icons  in  graphics-based  systems,  comparative  studies, 
system  development,  input  control  devices,  display  options,  training  issues, 
and  special  applications  for  graphical  interface  design  are  discussed. 

Early  Models 

In  1980,  Card,  Moran  and  Newell  examined  the  design  and  evaluation  of 
present-day  interactive  computer  systems.  The  authors  recognized  that 
users  differ  in  many  significant  ways.  Some  of  these  differences  included 
task  knowledge,  system  knowledge,  motor  skills,  technical  ability,  and 
experience.  Under  experience,  the  authors  listed  three  categories  of  users: 
novice,  casual,  and  expert.  The  authors  also  recognized  a  performance 
dimension,  including  separate  measures  of  time,  errors,  learning, 
functionality,  recall,  concentration,  fatigue,  and  acceptability  (4:396-397). 

Given  these  multiple  dimensions,  Card,  Moran,  and  Newell  focused  on 
the  amount  of  time  it  takes  expert  users  to  perform  routine  tasks.  From  this 
research,  the  authors  developed  their  “Keystroke-Level  Model,"  a 
mathematical  model  designed  to  allow  systems  designers  to  predict  the  time 
it  will  take  a  user  to  complete  a  task  using  the  given  system  (4:409).  Card, 
Moran,  and  Newell  recognized  limitations  to  their  model  in  that  it  only 
considered  expert  users,  routine  tasks,  and  error-free  performance.  In  their 
conclusion,  the  authors  addressed  these  limitations  in  calling  for  more 
research  considering  other  types  of  users,  tasks,  and  evaluative  measures 
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(4:409).  Some  of  today’s  most  popular  graphical  systems  have  incorporated 
these  early  concerns  into  their  design,  and  their  impact  on  all  types  of  users 
is  the  basis  for  the  research  in  this  thesis. 

In  1981,  Moran  built  on  this  early  research  by  stating  that  there  are  two 
ways  to  improve  user  performance.  The  first  option  was  to  alter  the  user 
interface,  thus  changing  the  task  structure.  The  second  option  was  to  teach 
the  user  more  efficient  methods,  thus  increasing  his  knowledge  (8:3). 
However,  Moran  concluded  on  a  somewhat  discouraging  note  for  advocates 
of  graphics-based  interfaces  by  stating  “there  is  a  surprising  number  of 
studies  that  show  only  small  differences  among  user  interfaces  for  expert 
users”  (20:9).  He  summarized  “these  results  signal  a  general  caution  that 
many  interface  features  have  less  effect  on  behavior  than  enthusiastic 
designers  imagine"  (20:9). 

Cognitive  Issues 

Carey  was  among  the  first  to  comment  on  the  correlation  of  interface  to 
training  costs  and  job  satisfaction.  Carey  hypothesized  that  managers’ 
avoidance  of  long  training  times  indicates  that  systems  designers  have  not 
been  able  to  offer  reinforcement  to  the  user  (following  minimal  learning)  to 
further  explore  systems  on  his  own.  Carey  also  stated  that  managers  are 
unsuccessful  when  they  try  to  force  unwelcome  systems  on  their  staffs.  He 
linked  stress  and  job  dissatisfaction  with  inadequate  systems  (5:15).  Carey 
concluded  that  in  designing  user-friendly  systems  (user-friendly  defined  as 
friendly  to  the  user’s  mode  of  working),  design  teams  should  incorporate  the 
following  roles:  applied  cognitive  psychologist,  linguist,  educator, 
cartographer,  and  consumer  marketing  consultant  (5:19).  Carey  recognized 
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that  like  the  parent  field  of  MIS,  interface  design  is  multi-disciplinary  in 
nature,  and  must  consider  and  apply  concepts  from  many  different  fields  to 
be  successful. 

Ives  examined  the  role  of  icons  in  the  design  of  computer  interfaces. 

Ives’  research  indicated  a  “very  low  entry  cost”  for  users  of  graphical 
systems  which  represent  familiar  objects  such  as  file  folders,  in/out  baskets, 
etc.  (14:36).  Ives  stated  that  because  graphical  systems  use  no  unique 
nomenclature  and  no  procedural  language,  no  extensive  training  program  is 
required  (14:37).  Ives  also  focused  on  the  relationship  of  icons  to  memory 
limitations  and  reported  that  the  use  of  graphics  does  not  interfere  with 
verbal  tasks.  Ives  referenced  research  by  Shepard  indicating  that  memory 
for  pictures  is  more  proficient  than  memory  for  words  (14:37).  Although 
Ives  did  not  state  it  directly,  he  implied  that  the  use  of  a  well-developed, 
icon-driven  interface  may  be  a  method  for  increasing  productivity  and 
user-friendliness  (14:36-37). 

Ives’  report  was  further  amplified  in  conclusions  reached  by  Norman  in 
1983.  In  his  research,  Norman  analyzed  what  he  termed  “slips."  The  author 
defined  a  as  a  situation  where  the  user’s  intention  was  proper,  but 
results  did  not  conform  to  that  user’s  intention  (22:254).  Norman  remarked 
that  the  user  has  a  mental  image  of  the  system  (a  graphical  representation) 
and  how  that  system  works.  He  believed  that  designers  must  be  cognizant 
of  the  user’s  image  in  system  design  (22:257). 

More  specifically,  Norman  argued  that  designers  should  incorporate  four 
factors  into  all  systems.  First,  the  system  should  provide  feedback  to  the 
user  by  allowing  the  state  of  the  system  to  be  clearly  available  at  all  times. 
Second,  there  should  be  a  similarity  of  response  sequences.  Norman  felt 
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that  different  classes  of  actions  should  have  separate  command  sequences. 
Third,  actions  should  be  reversible  when  possible;  and,  when  not  reversible, 
actions  should  be  difficult  to  accomplish,  thus  preventing  slips.  Finally, 
Norman  argued  that  the  system  should  incorporate  a  consistent  structure 
and  design  of  command  (22:257).  Norman  recommended  that  designers 
make  use  of  psychological  mechanisms,  recognizing  the  importance  of  the 
way  people  process  information.  Norman  believed  that  interface  design 
should  be  based  on  mental  models  of  the  user  and  that  designers  should 
analyze  the  types  of  errors  people  make,  and  design  the  interface  around 
these  types  of  errors  (22:257). 

Schneiderman  was  the  first  to  use  the  phrase  “direct  manipulation"  to 
describe  users’  interaction  with  objects  represented  on  a  computer  display 
terminal.  The  author  found  that  physical,  spatial  or  visual  representations 
are  easier  to  retain  and  manipulate— as  Ives  also  reported.  However, 
Schneiderman  cautioned  developers  because  he  found  that  graphic  icons 
which  present  a  cluttered  appearance  lose  their  usefulness.  He  also  warned 
designers  that  icons  should  have  a  clear  meaning,  and  should  not  occupy  an 
excessive  amount  of  screen  space  (24:64).  Schneiderman  concluded  that 
the  use  of  direct  manipulation,  especially  when  implemented  through 
graphics,  offers  exciting  possibilities  for  reducing  learning  time,  speeding 
performance,  and  increasing  satisfaction  (24:68). 

In  related  research,  Schneiderman  stated  that  using  psychologically- 
oriented  experimentation  is  the  best  approach  to  discovering  the  critical 
aspects  to  interface  design  and  provides  better  evidence  than  unsupported 
arguments  over  “user-friendliness."  He  felt  that  scientific  measurement  of 
learning  time,  performance  speed,  error  rates,  user  satisfaction,  and  memory 
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limitations  provide  invaluable  information  for  making  design  decisions  and 
choosing  between  competing  products  (25:3).  Schneiderman  made  the 
observation  that  systems  employing  “direct  manipulation"  create  excitement 
and  enthusiasm  from  users  and  lead  to  positive  feelings  of  system  mastery, 
competence  in  task  performance,  ease  in  learning  the  original  system  and  in 
learning  new  features,  confidence  in  the  ability  to  retain  system  mastery 
over  time,  eagerness  to  show  off  the  system  to  novices,  and  the  desire  to 
explore  the  more  powerful  aspects  of  the  system  (25:13-14).  Schneiderman 
argued  that  systems  employing  “direct  manipulation"  allow  novices  to  learn 
the  basic  system  quickly,  allow  experts  to  work  rapidly  and  easily 
implement  new  features,  and  reduce  the  anxiety  of  users  because  the 
system  is  understandable  and  actions  are  easily  reversible  (25:20). 

Herot  contended  that  ease  of  use  is  directly  related  to  the  way  in  which 
information  is  presented  to  the  user  (12:83).  Herot  was  a  proponent  of 
Schneiderman's  “direct  manipulation,"  but  used  the  term  “spatial  data 
management,"  to  describe  they  way  in  which  users  can  easily  organize, 
locate,  and  handle  a  variety  of  information  (12:84).  In  “spatial  data 
management,”  Herot  asserted  that  graphic  representations  are  the  most 
useful  way  to  present  information  (12:86). 

In  1985,  Bennett  commented  on  the  notion  of  a  mental  model,  as  first 
discussed  by  Card,  Moran,  and  Newell,  in  computer  systems  design  (2:4). 
Bennett  showed  that  users  who  interact  with  graphic  displays  form 
different  conceptual  and  mental  models  of  the  system  (2:5).  As  Davis 
suggested,  Bennett  felt  that  making  computer  systems  more  efficient  is  not 
enough— consideration  must  also  be  given  to  user  limitations.  Bennett 
stated  that  designers  must  look  at  cognitive  variables  to  improve 
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effectiveness  (2:12).  Bennett  remarked  that  the  use  of  graphics  is  important 
to  the  user  in  forming  the  mental  model  and  that  graphics-based  systems 
seem  “incredibly  easy  to  use”  (2;  12).  In  his  research,  the  author  illustrated 
that  relatively  small  changes  in  graphic  displays  can  have  a  large  impact  on 
user  performance  (2:13). 

Hollan  stressed  that  graphical  interfaces  have  important  cognitive 
properties.  First,  they  provide  a  physical  representational  system  which 
allows  users  to  better  understand  abstract  relationships  and  make  use  of  the 
brain's  powerful  pattern-matching  ability.  Second,  graphics-based  systems 
make  the  depiction  of  models  possible  that  are  similar  to  the  mental  models 
or  simulations  people  use  to  reason  about  the  world.  Third,  graphical 
interfaces  better  depict  physical  state  information  or  causal  connections  and 
allow  the  user  to  see  changes  in  the  state  of  the  system.  Finally,  graphical 
systems  provide  the  potential  of  “directly  manipulable  representations  of 
systems”  (13:7).  Hollan  stated  that  “interface  design  is  currently  much 
more  of  an  art  than  a  science"  (13:25).  Hollan  maintained  that  we  currently 
do  not  have  any  workable  theories  of  interface  design  and  therefore  do  not 
understand  what  contributes  to  the  effectiveness  of  most  successful 
interfaces  ( 1 3:25).  Hollan  contended  that  the  graphic  interface  creates  a 
dynamic  world  in  which  users  can  interact  with  representations  as  if  they 
were  real  world  objects  which  aids  the  user  in  working  with  the  system 
(13:25). 

The  Use  of  Icons  in  Graphics-Based  Systems 

More  recent  research  has  focused  on  the  use  of  icons  in  graphics-based 
interfaces.  Rogers  commented  on  this  trend  by  examining  the 
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representation  of  “low  imagery"  (or  abstract)  verbs  through  graphics.  She 
found  that  drawings  of  both  concrete  and  abstract  verbs  were  meaningful 
to  all  types  of  users.  As  one  might  expect.  Rogers  found  that  drawings  for 
the  “high  imagery”  verbs  were  more  representative  of  a  particular  action 
than  were  “low  imagery”  verbs.  Users  were  also  able  to  match  specific 
commands  with  icon  sets;  though  again,  those  icons  depicting  concrete 
objects  had  the  most  matches.  Therefore.  Rogers  stated  that,  for  some 
commands,  the  pictorial  form  of  the  icon  is  critical  (23:43-44).  The  author 
concluded  that  there  is  potential  for  using  icons  to  represent  commands 
without  prior  learning.  Rogers  then  suggested  further  research  on 
evaluating  the  advantages  of  icons  versus  existing  command  sets  (23:44). 
Her  suggestion  is  one  of  the  primary  objectives  of  this  thesis. 

Similarly,  Gittins  found  that  icon-based  interfaces  “provide  more  usable 
dialog  (than  command-driven  interfaces)  because  of  their  capacity  to  carry 
much  greater  descriptive  information"  (9:519).  He  criticized  undue 
emphasis  on  the  original  “office  metaphor”  stating  that  such  emphasis  shows 
a  “lack  of  originality  and  absence  of  systematic  design”  (9:521).  Gittins  then 
listed  a  number  of  advantages  and  disadvantages  of  icon-driven  interfaces. 

Among  the  advantages,  Gittins  stated  that  the  user  can  infer  function 
from  pictographic  symbols.  Second,  he  reported  that  icons  aid  in  reducing 
errors.  Users  are  more  accurate  because  they  are  able  to  choose  from  a 
given  set  of  choices  and  are  not  reliant  on  their  memory  of  valid  commands. 
Third,  Gittins  stated  that  icons  can  illustrate  commonality  between  similar 
concepts.  Fourth,  as  Schneiderman  first  described,  Gittins  reported  that 
icons  facilitate  direct  manipulation  of  items  on-screen.  Last.  Gittins  stated 
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that  designers  can  get  more  information  to  the  user  in  a  smaller  space— in 
the  cognitive  sense  as  well  as  the  literal  sense  (9:526-527). 

Among  the  disadvantages.  Gittins  found  that  it  was  difficult  to  represent 
some  functions  with  icons,  though  it  did  help  to  add  words  to  those 
particular  icons.  This  finding  supported  Rogers'  study,  though  Rogers  still 
found  potential  for  using  icons  to  represent  abstract  concepts  (23:43-44). 
Gittins  also  found  that  intricate  desigri  and  color  can  complicate  (rather 
than  simplify)  user  dialog.  Finally,  the  author  stated  that  it  is  extremely 
difficult  to  establish  any  measure  of  “usability"  for  icon-based  systems 
(9:527-529). 

Johnson  complemented  Gittins*  with  his  argument  that  computers 
should  not  try  to  approximate  the  actual  office.  Johnson  felt  that  the  use  of 
the  “desktop  metaphor"  in  graphical,  direct  manipulation  interfaces  was 
flawed  ( 16:2 1 ).  Johnson  argued  that  the  goal  of  making  a  system  easy  to 
learn  through  the  use  of  familiar  objects  is  lost  when  designers  try  to 
simulate  the  desktop  in  too  literal  terms.  Johnson  believed  that  most 
systems  do  not  recognize  that  manipulating  objects  on-screen  is  often  more 
difficult  than  actually  manipulating  those  objects  in  the  physical  world.  He 
felt  that  systems  using  the  “desktop  metaphor”  create  unnecessary 
development  effort  and  poor  system  performance.  Similarly.  Johnson  stated 
that  these  systems  often  limit  electronic  functionality  to  that  of  the  physical 
object  that  is  represented,  and  do  not  address  a  “better"  way  of  doing  things 
(16:22).  Finally,  Johnson  commented  that  though  this  type  of  interface  is 
familiar,  it  is  not  necessarily  optimal.  He  stated  that  there  are  no  rules  for 
how  detailed  the  metaphor  should  be  (16:23).  Johnson  concluded  by  stating 
that  deviations  from  the  “desktop  metaphor"  are  often  necessary.  He  stated 
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that  unless  designers  understand  when  and  why  to  simulate  the  real  office, 
they  are  likely  to  end  up  with  a  sophisticated  product  that  is  difficult  to  use 
(16:23). 

Guastello,  Traut,  and  Korienek  completed  the  most  recent  research  on 
the  use  of  icons  and  published  their  results  in  July  1989.  The  authors  listed 
three  important  considerations  for  systems  designers.  First,  icons  which 
mix  pictures  and  words  are  more  meaningful  than  verbal  or  pictorial  icons 
only.  Second,  the  authors  found  that  the  validity  of  using  “mixed  modality* 
icons  (combining  verbal  and  pictorial  representations)  is  true  across  diverse 
content  areas,  including  engineering,  data  processing,  and  financial  arenas. 
Finally,  Guastello,  Traut,  and  iCorienek  reported  that  it  is  possible  to  design 
new  icons  that  are  more  meaningful  than  current  industry  standards 
(11:119).  The  authors  premise  was  that  interface  design  must  be  based  on 
data  obtained  from  the  user  population  (1 1:99).  The  authors  concluded  that 
the  survey  method  is  the  most  appropriate  way  to  obtain  this  data  and 
apply  it  to  the  study  of  human-computer  interaction  (11:1 19). 

Comparative  Studies 

John  Whiteside  examined  the  performance  and  reactions  of  76  users 
with  seven  different  commercial  interfaces  representing  command,  menu 
and  iconic  interfaces.  In  his  tests,  the  basis  for  comparison  among  the 
interfaces  was  user  performance  and  preference  on  a  specially-designed, 
standardized  task  that  stressed  simple  filing  operations  (27:185). 
Specifically,  Whiteside  attempted  to  answer  3  questions: 

1.  Are  there  large  user  performance  differences  between  interfaces? 

2.  What  style  of  interface  is  best  for  what  level  of  user? 
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3.  Can  performance  differences  be  attributed  to  interface  style? 
(27:185) 

In  evaluating  these  questions  Whiteside  made  the  point  that  each  type  of 
interface  was  tested  as  a  whole  system.  That  is,  no  attempt  was  made  to 
isolate  the  mode  of  presentation,  the  terminal  used,  or  the  keyboard  used. 
The  author  claimed  that  the  entire  system  represents  a  design  team's  best 
solution  to  optimizing  many  variables;  therefore,  it  is  more  appropriate  to 
test  the  whole  system  (27:185).  Whiteside  presented  the  user  with  minimal 
information  on  each  system  ahead  of  time,  but  the  user  did  receive  the 
documentation  for  each  system.  Users  were  then  required  to  complete 
basic  file  manipulation  procedures  on  text  files  including  displaying, 
merging,  and  sending  to  other  users.  Results  were  recorded  using 
performance  measures,  videotapes,  interviews,  and  questionnaires.  The 
researcher  controlled  the  task  performed  and  the  users’  computer 
background  (27:186-187). 

The  results  of  Whiteside’s  study  were  surprising.  He  found  that  new 
users  performed  better  on  command  and  menu  systems  than  on  iconic 
systems.  He  also  found  that  one  icon  system  was  much  preferred  over  a 
second  icon  system.  There  were  also  feedback  problems  noted  on  all 
systems  tested.  On  the  command  system,  users  became  frustrated  due  to 
the  inconsistency  of  function  key  operations  across  all  applications.  Finally, 
Whiteside  noted  difficulty  with  using  the  mouse  on  icon  systems  due  to  that 
input  device’s  perceived  lack  of  reliability  (from  the  user’s  perspective) 
(27:187-189). 

Overall,  Whiteside  noted  large  differences  in  the  performance  of  what 
the  author  called  new,  transfer,  and  system  users.  There  were  also  large 
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differences  between  systems;  however,  these  differences  were  not 
dependent  on  the  user’s  experience  level  (new,  transfer,  or  system).  After 
also  considering  user’s  qualitative  ratings  for  each  system,  in  addition  to  the 
performance  measures  outlined  above,  Whiteside  concluded  that  the 
menuing  system  was  the  worst— especially  for  new  users.  The  author 
stated  that  this  was  surprising  because  menu  systems  are  typically 
designed  to  ease  computing  for  new  users.  In  addition,  no  icon  system 
allowed  the  beginner  to  produce  useful  work  within  a  half  hour  unassisted. 
Whiteside  also  discovered  that  there  was  no  evidence  that  making  systems 
easy  for  beginners  makes  those  same  systems  difficult  for  experts.  In  fact, 
Whiteside’s  results  suggest  that  making  systems  easy  to  use  for  beginners 
also  improves  ease  of  use  for  experts.  This  result  is  also  counterintuitive  to 
conventional  arguments  on  interface  design  (27:189-190).  In  summing  up 
his  results,  Whiteside  stated  that  interface  style  did  not  emerge  as  an 
important  predictor  of  performance  or  preference  on  different  systems. 
Second,  new  interface  styles  do  not,  by  themselves,  solve  old  human  factors 
problems.  Some  problems  were  common  across  all  three  types  of  interfaces. 
Finally,  the  author  concluded  that  rather  than  the  type  of  interface,  the  care 
with  which  it  is  designed  and  built  is  the  critical  factor  in  the  success  of 
that  system  and  that  this  comes  with  product  maturity  (27:190). 

Whiteside’s  study  is  important  because  it  refutes  many  traditional 
arguments  for  specific  types  of  interfaces  with  empirical  evidence  to  the 
contrary.  His  results  indicate  a  need  for  further  study  in  this  area  to 
determine  what  impact,  if  any,  the  interface  has  on  different  types  of  users. 

In  contrast  to  Whiteside’s  study.  Diagnostic  Research,  Inc.  conducted 
two  studies  in  1988  among  Macintosh  users  and  MS-DOS  users  and  found 
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that  users  rated  the  Macintosh  system  higher  in  virtually  all  areas  that  were 
measured  in  the  study.  One  study  was  conducted  among  MIS  managers 
familiar  with  both  Macintosh  and  MS-DOS  systems.  The  second  study 
consisted  of  Fortune  1000  professionals  from  other  departments  who 
regularly  used  either  Macintosh  or  MS-DOS  personal  computers  in  their 
work.  Issues  examined  were  user  productivity,  ease  of  use,  and  training 
time  (19:1). 

Overall,  Diagnostic  Research’s  results  indicated  the  Macintosh  has 
advantages  over  MS-DOS  systems.  In  the  productivity  category,  the 
Macintosh  was  judged  easier  for  learning  the  system  and  for  learning  new 
applications  (19:3).  In  training  time  and  costs,  users  reported  learning  the 
Macintosh  twice  as  fast  as  MS-DOS,  cutting  training  costs  in  half.  MIS 
managers  reported  a  mean  training  cost  of  approximately  $300  for 
Macintosh  users  compared  with  $750  for  MS-DOS  users.  Approximately  20 
hours  of  training  time  was  required  per  MS-DOS  user  versus  9  hours  for 
Macintosh  users  (19:5-6).  In  support  time,  managers  reported  that  MS-DOS 
systems  required  approximately  31  hours  per  month  per  computer  as 
compared  with  only  15  hours  for  Macintosh  systems  (19:7).  The  Macintosh 
also  received  higher  ratings  on  the  quality  of  business  graphics  and  the 
quality  of  printed  output  (19:8).  Macintosh  users  also  reported  that  they 
were  familiar  with  57  percent  more  software  applications  than  MS-DOS 
users  (19:9).  Finally,  users  of  the  Macintosh  rated  their  system  higher  than 
MS-DOS  users  for  being  enjoyable  to  use  and  giving  them  confidence  in 
using  their  computer.  MIS  managers  also  gave  the  Macintosh  higher  ratings 
for  end  user  satisfaction  (19:10). 
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These  results  are  not  what  one  would  expect  after  examining  the  results 
of  Whiteside  s  study.  The  results  of  Diagnostic  Research’s  study  are  the 
most  definitive  evidence  to  date  endorsing  the  use  of  a  particular  graphical 
user  interface.  However,  when  compared  with  similar  research  in  the  field, 
there  is  still  disagreement  on  exactly  what  impact  the  interface  has  on 
measures  of  user  productivity. 

System  Development 

One  of  the  major  reasons  that  graphical  user  interfaces  have  been 
relatively  slow  in  gaining  acceptance  is  the  difficulties  they  pose  for 
developers.  Though  often  viewed  as  easy  to  use  from  the  user’s 
perspective,  from  the  developer’s  perspective  graphical  user  interfaces  are 
usually  more  difficult  to  design.  Because  of  their  design  difficulty,  graphical 
user  interfaces  are  more  costly  to  develop,  thus  keeping  many  major 
developers  out  of  the  graphical  market. 

Wixon  and  Whiteside  emphasized  that  development  of  a  system  which 
is  easy  to  use  does  not  occur  from  simply  good  intentions  or  a  lot  of  money 
for  development  (28:144).  Wixon  and  Whiteside  developed  a  project  known 
as  the  User-Derived  Interface  (UDI)  which  attempted  to  build  a  system 
which  allows  novices  to  perform  simple  tasks  in  one  hour  without 
consulting  a  manual,  on-line  help,  or  a  tutorial.  In  developing  the  UDI,  the 
authors  found  that  “usability”  was  difficult  to  define.  Wixon  and  Whiteside 
listed  two  alternative  definitions  of  “usability."  The  first  definition  was  a 
system  which  supports  high  levels  of  productivity  for  those  who  use  that 
system  continuously  after  much  training.  The  second  definition  was  a 
system  which  allows  the  user  to  complete  productive  work  within  the  first 
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half  hour  without  any  training  (28:145).  In  summing  up  their  work  on  the 
UOI  project,  the  authors  commented  on  the  importance  of  setting  levels  of 
usability  to  be  achieved  for  “success”  in  advance.  Secondly,  Wixon  and 
Whiteside  advocated  taking  a  prototyping  approach  to  developing  an 
interface  for  usability.  The  authors  felt  that  a  program  of  iterative  testing 
with  very  short  development  and  test  cycles  best  meets  the  needs  of  users 
(28:145-146).  Finally,  and  most  importantly,  Wiion  and  Whiteside  believed 
that  building  usability  into  an  interface  design  requires  a  systematic, 
engineering  process  (28:147). 

(Ceith  A.  Butler  also  described  the  importance  that  the  design  stage  plays 
in  developing  effective  user  interfaces.  Butler  defined  “learnability”  as  a 
key  attribute  because  of  its  importance  to  the  user’s  perception  of  system 
quality  (3:85).  The  author  described  a  system  for  designing  usable  systems 
based  on  empirical  definitions  of  desirable  system  performance. 

Specifically,  Butler  advocated  specifying  user  performance  objectives  as 
part  of  the  product  definition  stage,  applying  human  factors  principles  and 
methods  during  the  design  and  implementation  stage  of  product 
development,  and  performing  testing  of  user  performance  to  evaluate 
whether  the  product  meets  user  performance  objectives  specified  in  the 
product  definition  stage  (3:85).  Butler  stated  that  there  are  three  main 
dimensions  of  performance  that  comprise  usability:  learnability, 
throughput,  and  user  attitude.  He  also  stated  that  one  can  define  “user- 
friendliness"  by  quantifying  the  probability  of  task  completion  (3:86).  To 
test  his  theory,  Butler  gave  subjects  a  problem  and  left  them  with  a  users 
guide  to  software  developed  using  his  three  development  principles. 

Butler’s  main  contention  was  that  in  interactive  systems,  just  meeting 
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performance  criteria  is  not  enough — the  developer  must  also  consider  user 
performance  criteria  to  increase  productivity  (3:88).  After  testing,  Butler 
found  that  his  systematic  approach  does  offer  impressive  performance.  In 
his  test,  subjects  completed  problems  in  an  average  of  123.8  minutes 
compared  with  180  minutes  which  was  defined  as  the  learnability  objective 
(3:88).  Butler’s  results  lend  further  credence  to  the  importance  of  defining 
usability  objectives  in  advance  of  the  design  effort  and  suggests  that 
systems  which  skip  this  design  stage  may  suffer  unforeseen  problems  with 
user  acceptance  or  learning. 

In  evaluating  the  effectiveness  of  a  particular  design  effort,  Michelle  A. 
Lund  recommended  a  prototyping  approach  for  design  teams.  Lund 
suggested  that  when  designing  an  interface  for  graphics  applications,  as 
portions  of  the  design  phase  are  completed,  give  potential  users  a  task  to 
complete  then  observe  and  videotape  the  results.  Her  thesis  was  that 
although  developers  can  (and  should)  test  their  efforts,  there  is  no  substitute 
for  human  factors  testing  with  real  users.  By  testing  in  this  manner, 
developers  can  discover  surprises  about  likes  and  dislikes.  The  prototyping 
approach  then  allows  designers  to  make  changes  while  still  in  the 
development  stage  before  releasing  the  product  to  the  field  (18:107).  Lund 
stated  that  interface  testing  should  include  several  elements.  First,  the 
testing  should  determine  how  well  a  system  anticipates  a  user’s  train  of 
thought.  Second,  when  the  user  gets  lost,  testing  should  reveal  what  led  the 
user  in  the  wrong  direction.  Third,  testing  should  discover  problems  the 
first  time  a  user  encounters  them — before  the  user  becomes  accustomed  to 
it.  Fourth,  the  test  should  document  how  the  user  reacts  to  specific  design 
features  (such  as  help  screens,  etc.).  Finally,  the  testing  should  indicate 
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what  concepts  need  to  be  stressed  in  training  and  in  the  documentation  for 
the  system  (18:109).  Lund  felt  that  the  best  way  to  test  users  is  to  give 
them  a  problem,  having  them  vocalize  all  thoughts  as  they  try  to  solve  this 
problem  with  the  system.  The  evaluator  then  videotapes  the  entire  session 
to  capture  the  user's  reactions  to  the  system  in  a  given  situation.  Lund 
stated  that  she  finds  this  evaluation  method  highly  effective  because  it 
successfully  identifies  problems  in  the  user  interface,  and  it  allows 
designers  to  verify  that  design  changes  have  improved  the  system  under 
development  upon  re-test  (18:112-113). 

Input  Control  Devices  for  Graohics-Based  Systems 

Since  virtually  all  graphical  interfaces  are  mouse-driven,  research  into 
the  features  of  various  input  devices  is  an  important  consideration.  Embley 
and  Nagy  examined  different  behavioral  aspects  of  text  editors.  One 
category  that  the  authors  examined  as  part  of  their  study  was  input  devices. 
Using  the  aforementioned  Card  model,  Embley  and  Nagy  reported  that  “the 
mouse  is  a  uniformly  superior  cursor  control  device  with  respect  to  both 
speed  and  error  rate"  (8:62).  However,  for  some  tasks,  the  authors 
reccmmended  that  keyboard  control  be  retained  (8:62). 

On  the  other  hand,  a  study  conducted  in  1986  by  ICarat,  McDonald,  and 
Anderson  contradicted  the  findings  of  Embley  and  Nagy.  Compared  to  the 
touch  panel  and  keyboard,  the  mouse  was  the  least  preferred  input  control 
device  ( 17:8 1 ).  Recognizing  the  studies  of  Embley  and  Nagy  (as  well  as 
others),  ICarat,  McDonald,  and  Anderson  listed  several  limitations  to  their 
research  which  may  have  had  an  impact  on  their  findings.  First,  the 
subjects  used  in  their  first  set  of  experiments  were  all  skilled  typists. 
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Second,  the  mouse  was  a  new  device  for  all  the  subjects  tested  and  was 
therefore  less  familiar  than  either  touch  panels  or  keyboards  (17:74).  In 
their  conclusion,  the  authors  stated  that  despite  their  findings  reporting  a 
preference  for  touch  panels  as  an  input  control  device,  the  mouse  offers  the 
advantage  of  allowing  programmed  functions  to  be  built  into  the  mouse 
buttons — an  advantage  not  possible  with  touch  panels  (17:88). 

Display  Potions 

Grudin  contradicted  the  more  popular  argument  for  interface 
consistency  offered  by  Schneiderman  and  others  (10:1 164).  Whereas 
Schneiderman  stated  that  maintaining  consistent  properties  leads  to  better 
interface  design,  Grudin  argued  that  interface  consistency  is  unworkable. 
While  interface  consistency  can  contribute  to  ease  of  learning,  Grudin 
argued  that  ease  of  learning  often  conflicts  with  ease  of  use— an  argument 
which  Whiteside  rejected  in  his  research  (27:189).  Particularly  for  skilled 
users.  Grudin  believed  that  interface  consistency  works  against  good  design 
( 10:1 166).  The  author  believed  that  as  task  knowledge  increases, 
consistency  becomes  less  important  and  maintaining  consistency  across  a 
wide  range  of  applications  and  types  of  users,  like  the  Macintosh,  leads  to 
increasing  performance  costs.  Like  Johnson.  Grudin  felt  that  adhering  to 
real  world  objects  in  designing  interfaces  does  not  make  the  best  use  of 
available  technology  (10:1 166).  Grudin’s  solution  in  designing  an  interface 
was  to  fully  know  and  understand  the  intended  users  and  tasks  better  than 
anyone  else  (10:1 172).  Grudin’s  research  is  significant  because  it  is  one  of 
the  only  published  works  in  the  field  which  argues  against  maintaining 
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interface  consistency  and  suggests  that  the  look  and  feel  of  the  interface 
may  not  be  as  important  as  once  thought. 

Daniel  J.  Dwyer  examined  the  effect  that  screen  size  has  on  graphical 
displays  and  job  performance.  In  his  experiments,  Dwyer  tested  screens 
with  5"  X  5".  9“  X  9".  and  12"  x  12"  displays.  After  conducting  his  tests,  the 
researcher  concluded  that  screen  size  has  an  effect  on  response  accuracy 
only  when  discriminability  between  test  points  is  low  (7:1 1).  Dwyer  found 
that  when  a  graphic  display  is  composed  of  many  similar  densely-packed 
points,  these  points  tend  to  compete  for  a  person’s  attention  on  smaller 
screen  sizes.  However,  when  the  ability  to  discriminate  between  points  is 
relatively  easy,  accuracy  fluctuates  only  slightly  (7:1 1).  Finally,  Dwyer 
found  no  impact  of  screen  resolution  on  response  accuracy  (7:12).  There  is 
currently  no  literature  indicating  any  link  between  resolution  and  screen 
size.  These  findings  suggest  that  arguments  against  some  graphics-based 
systems  which  use  small  screen  sizes,  like  the  Macintosh,  may  not  be  valid. 
On  the  other  hand,  many  graphics-based  systems  counter  the  argument 
against  small  screen  size  with  their  argument  for  superior  resolution.  Again, 
Dwyer’s  results  refute  this  argument  and  suggest  that  screen  resolution 
may  not  have  an  impact  on  productivity. 

Training  and  Learning  Issues 

Ives,  Olson,  and  Baroudi  stated  that  most  MIS  failures  come  from  a  lack 
of  user  acceptance  of  that  system.  They  also  found  that  a  system  which 
meets  the  needs  of  its  users  will  reinforce  satisfaction  with  that  system. 
Conversely,  a  system  which  fails  to  meet  the  needs  of  its  users  will 
reinforce  dissatisfaction  with  that  system  (15:785-787).  Ives.  Olson,  and 
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Baroudi’s  results  have  challenged  organizations  to  help  increase  user 
satisfaction.  One  of  the  ways  which  organizations  have  attempted  to  do  this 
is  through  structured  training  programs  designed  to  increase  familiarity 
with  the  system  and  promote  user  acceptance  of  that  system. 

Following  Ives.  Olson,  and  Baroudi's  findings.  Nelson  and  Cheney 
conducted  a  field  study  of  100  middle  and  upper  level  managers  in  20 
companies.  In  the  authors'  literature  review,  they  found  a  lack  of  consensus 
on  whether  training  was  the  critical  factor  in  getting  managers  to  use 
information  systems  technology  (21:556).  In  their  survey.  Nelson  and 
Cheney  found  that  respondents  reported,  almost  exclusively,  that  they  felt 
training  was  important  to  the  successful  integration  of  information  systems 
products.  However,  the  researchers  also  found  that  only  two  percent  of 
information  systems  resources  were  spent  on  training  end  users  (21:556). 
From  their  results  in  the  20  organizations  researched,  Nelson  and  Cheney 
found  that  there  is  a  positive  relationship  between  the  computer  training 
that  a  user  receives  and  the  user's  ability  to  use  the  computer.  In  addition, 
the  authors  found  that  there  is  a  positive  relationship  between  the  user's 
ability  and  their  utilization  and  satisfaction  with  information  systems 
technology  (21.554-555).  From  these  results.  Nelson  and  Cheney  concluded 
that  information  systems-related  education  and  training  leads  to  acceptance 
and  usage  of  information  systems  technologies  throughout  the  organization 
(21:547). 

John  C.  Thomas  of  the  IBM  Research  Center  also  believed  that  interface 
design  is  critical  to  the  success  of  today’s  computer  user.  Thomas 
emphasized  that  early  computer  users  were  dedicated,  technical  specialists. 
As  hardware  costs  have  fallen,  the  installed  user  base  has  broadened  to 
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include  all  types  of  users.  These  two  factors  have  led  to  a  much  greater 
importance  on  the  ease  with  which  computer  systems  can  be  learned  and 
used.  Observations  at  the  IBM  Research  Center  indicate  that  it  is  unlikely 
that  new  users  will  sit  down  and  read  a  100  page  manual.  Thomas  found 
that  users  are  more  likely  to  skip  around  and  try  out  different  functions 
(26:3 1 ).  Thomas  stated  that  individual  ability  and  attitude  toward  the 
computer  system  tremendously  affect  productivity  (26:32).  For  office 
personnel,  Thomas  contended  that  ease  of  use,  ease  of  learning,  enjoyability, 
and  the  perceived  effect  on  productivity  will  have  a  great  impact  on 
computer  sales  to  the  population  in  question  (26:34).  In  addition,  the  author 
postulated  that  the  more  difficult  a  system  is  to  use.  the  higher  marketing 
costs  will  be  and  the  higher  service  costs  for  that  system  will  be  (26:35). 
Thomas  sunimarized  by  stating  that  only  by  designing  and  organizing  for 
human  factors  considerations  can  these  limitations  be  overcome  (26:45). 

Special  Aoplications  for  Graphical  Interface  Design 

Unlike  Grudin.  Adcock  addressed  a  need  for  a  single,  unified  user 
interface  which  allows  database  users  to  accomplish  all  activities  within  a 
single  environment.  Adcock  believed  that  a  graphical  interface  presents  the 
greatest  potential  (1:12).  Among  the  advantages  of  this  type  of  system, 
Adcock  stated  that  the  increased  dimensions  offered  by  pictures,  the  ease  of 
learning  and  transferring  knowledge  through  pictures,  and  the  increased 
ability  to  represent  the  real  world  are  not  found  in  any  other  system 
available  (1:12).  Adcock  outlined  a  database  system  with  a  graphical 
interface  to  best  fulfill  the  properties  of  descriptiveness,  power,  and  ease  of 
use  and  learning.  Adcock  remarked  that  the  graphical  interface  is 
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necessary  in  today’s  environment  due  to  the  growing  diversity  of  the  user 
base  ( 1 :46).  His  database  system  was  designed  to  facilitate  ease  of  learning 
through  use  of  a  consistent,  graphics-based  operating  environment  (1:46). 

Summary 

Based  on  the  review  of  the  literature,  there  appears  to  be  agreement 
that  the  diffusion  of  computer  systems  to  a  wider  base  of  users  has  created 
a  need  for  systems  that  are  easy  to  learn  and  easy  to  use.  Likewise, 
research  indicates  that  organizations  which  place  emphasis  on  training 
users  are  more  likely  to  successfully  implement  information  systems 
technologies.  However,  the  effect  that  graphical  user  interfaces  have  on 
this  process  is  not  clear.  Many  authors,  including  Schneiderman.  Norman, 
Hollan,  and  others  have  stated  that  graphical  systems  are  effective  at 
facilitating  end-user  computing.  However,  another  somewhat  smaller 
group  of  authors,  including  Grudin  and  Whiteside,  have  argued  that  the 
interface’s  role  in  creating  workable,  easy-to-use  systems  is  not  as 
significant  as  one  might  eipect.  There  is  evidence  to  suggest  that  the  type 
of  user  and  the  type  of  task  involved  may  determine  the  appropriateness  of 
a  particular  interface  for  a  given  environment.  To  date,  no  research  has 
been  conducted  in  Air  Force  offices  to  determine  if  graphical  user 
interfaces  have  any  impact  on  job  efficiency  and  effectiveness. 


26 


Ill.  Mettodoiogy 


This  chaptei  describes  the  methodology  that  was  used  to  accomplish 
the  research  objectives  and  to  answer  the  research  and  investigative 
questions  listed  in  Chapter  1  of  this  study.  Selection  of  the  methodology, 
the  population  from  which  the  data  was  collected,  the  survey  instruments 
which  were  used  to  collect  data,  and  the  statistical  tests  which  were  used  to 
analyze  the  data  are  described. 

Selection  of  Methodology 

The  survey  method  was  chosen  to  solve  the  research  problem.  There 
were  important  advantages  to  the  survey  method  used  in  this  research. 

First,  it  allowed  the  researcher  to  gather  results  from  a  large  sample 
population.  The  number  of  individuals  sampled  in  this  study  would  not 
have  been  feasible  with  any  other  method.  Because  the  survey 
methodology  allowed  the  researcher  to  gather  data  from  a  large  number  of 
individuals,  the  researcher  lelt  that  external  validity  was  enhanced  over 
other  potential  methodologies  for  this  research.  Finally,  the  use  of  a 
questionnaire  with  pre-defined  rating  scales  facilitated  statistical  analysis  of 
results  allowing  the  researcher  to  easily  compare  separate  populations. 

However,  one  distinct  disadvantage  was  also  considered  before 
choosing  the  survey  methodology.  Because  no  survey  instruments  were 
available,  a  written  questionnaire  was  constructed  to  answer  the 
investigative  questions  listed  in  Chapter  1.  To  help  improve  internal 
validity,  the  questionnaire  was  submitted  to  a  panel  of  experts  on  computer 
systems  and  survey  instruments  to  insure  that  undesirable  psychometric 
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qualities  were  eliminated.  By  taking  these  precautions,  the  researcher  felt 
confident  that  the  benefits  afforded  by  quantitative  analysis  of  a  large  user 
population  outweighed  any  potential  disadvantages  to  the  survey 
methodology. 

Population 

There  were  two  populations  of  interest  for  this  study.  The  first 
population  consisted  of  users  of  MS-DOS  (primarily  Zenith  Z-248)  computer 
systems  within  the  Advanced  Tactical  Fighter  Special  Programs  Office  and 
Wright  Research  and  Development  Center  located  at  Wright-Patterson 
AFB,  OH.  The  second  population  consisted  of  users  of  the  Apple  Macintosh 
computer  system  within  the  same  organizations.  Users  in  each  population 
were  limited  to  computer  users  who  used  their  computer  at  least  three 
hours  per  week,  had  used  the  particular  operating  system  for  three  months 
or  more,  and  were  familiar  with  more  than  one  software  application.  These 
qualifications  insured  that  ail  individuals  sampled  had  an  adequate  level  of 
knowledge  about  the  system  to  effectively  evaluate  that  system’s 
contribution  to  each  of  the  performance  measures. 

A  non-probability  judgement  sample  was  chosen  for  the  research.  That 
is.  only  individuals  meeting  the  criteria  specified  above  were  surveyed. 

The  researcher  felt  that  this  sampling  plan  was  the  best  alternative  and 
allowed  him  to  best  measure  the  variables  being  researched  in  this  study. 
The  purposive  sample  allowed  the  researcher  to  isolate  two  distinct 
populations  and  eliminate  intervening  variables  such  as  organizational 
environment  and  computer  experience  (or  lack  thereof)  on  a  particular 
system  being  studied.  However,  care  must  be  taken  in  over-generalizing 
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the  results  of  this  research  to  organizations  with  dissimilar  structure  or 
demographics. 

Survey  Instrument 

The  investigative  questions  used  in  this  study  were  as  follows: 

1 .  How  does  each  type  of  interface  compare  in  the  amount  of  time 
required  to  learn  the  basic  system? 

2.  How  does  each  type  of  interface  compare  in  the  amount  of  time  users 
report  is  typically  required  to  learn  a  new  application  program? 

3.  How  does  each  type  of  interface  compare  in  users’  ratings  of  user- 
friendliness? 

4.  How  does  each  type  of  interface  affect  users’  perceptions  of  the 
eitent  that  their  computer  system  helps  them  perform  in  their  job? 

5.  How  does  each  type  of  interface  affect  the  number  of  software 
packages  users  of  that  system  use  on  the  job? 

6.  How  does  each  type  of  interface  affect  user  satisfaction  with  their 
computer  system? 

7.  How  does  each  type  of  system  compare  in  responsible  authorities’ 
ratings  of  quality  of  output? 

8.  What  is  the  nature  of  the  relationship  between  users’  job  experience 
level  and  type  of  interface  used  on  the  job? 

A  survey  questionnaire  (see  Appendix)  was  constructed  to  gather  data 
and  appropriate  measurements  for  the  questions  outlined  above.  For  the 
purpose  of  this  study,  all  responses  were  considered  to  be  at  least  interval- 
level  data. 
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To  improve  internal  validity  of  the  instrument,  the  questionnaire  was 
submitted  to  a  panel  of  experts  in  both  information  systems  and  survey 
design.  Members  of  this  panel  included  Captain  Carl  L.  Davis,  Assistant 
Professor  of  Research  Methods  and  Technical  Communication;  Dr.  Charles 
R.  Fenno,  Associate  Professor  of  Research  Methods  and  Technical 
Communication;  Lieutenant  Colonel  Dorothy  J.  McBride,  Program  Manager 
for  the  Information  Resource  Management  graduate  program:  Lieutenant 
Colonel  Richard  E.  Peschke,  Assistant  Professor  of  Logistics  Management; 
and  Major  L.  Maurice  Riggins,  Instructor  in  Logistics  Management.  Panel 
expertise  was  consolidated  and  both  content  and  structural  changes  to  the 
questionnaire  were  incorporated  before  final  submission  of  the  survey 
instrument  to  the  Commander,  Aeronautical  Systems  Division  for  approval. 

The  survey  questionnaire  was  divided  into  seven  sections.  Section  1 
consisted  of  demographic  questions,  including  questions  designed  to  screen 
individuals  answering  the  questionnaire  based  on  the  criteria  listed  above 
for  the  judgement  sample  used  in  this  study.  In  addition,  the  information 
gathered  in  Section  1  was  used  in  the  contingency  table  analysis  used  to 
answer  Investigative  Question  8. 

Section  2  was  designed  to  answer  Investigative  Question  1 .  It  was 
based  in  principle  on  research  first  conducted  by  Carey,  Ives,  and 
Schneiderman  (5:14,  14:15,  25:57).  This  section  first  asked  respondents 
whether  they  knew  how  to  accomplish  a  set  of  basic  file  operations 
common  to  each  type  of  interface.  Then,  users  were  asked  how  much  time 
it  took  them  to  learn  to  perform  all  of  the  basic  file  operations  listed  in  the 
questionnaire.  Using  the  file  operations  listed  in  the  questionnaire  as  a 
benchmark,  the  researcher  was  then  able  to  draw  conclusions  about  how 
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much  time  was  required  to  learn  the  basic  system  for  each  type  of 
interface. 

Section  3  of  the  questionnaire  was  constructed  to  answer  Investigative 
Questions  2  and  5.  Diagnostic  Research’s  analysis  of  Macintosh  and  MS- 
DOS  computer  users  formed  the  basis  for  questionnaire  design  for  this 
portion  of  the  study  ( 19:1 ).  The  questionnaire  asked  users  whether  they 
used  their  computer  for  specific  types  of  applications.  Based  on  the 
organizations  surveyed,  the  researcher  asked  users  about  word  processing, 
spreadsheet,  database,  graphics/presentation,  and  engineering  applications. 
If  users  responded  that  they  did  use  their  computer  for  a  specific  type  of 
application,  they  were  then  asked  how  much  time  it  took  them  to  learn  to 
use  that  type  of  application  on  their  computer.  The  researcher  was  then 
able  to  compare  how  much  time  users  reported  was  required  to  learn  new 
applications  for  each  type  of  interface.  Finally,  the  researcher  asked 
respondents  how  many  application  programs  they  felt  proficient  on.  The 
results  from  this  item  as  well  as  the  responses  to  the  application-specific 
questions  yielded  data  used  to  answer  Investigative  Question  5. 

Section  4  was  designed  to  answer  Investigative  Question  3.  This 
section  was  designed  based  on  the  ideas  presented  by  Gittins,  Herot,  and 
Johnson  (9:519,  12:83, 16:21).  Users  were  first  presented  with  a  seven- 
point  Likert  scale  and  were  asked  to  use  the  scale  to  respond  to  a  series  of 
specific  questions  about  working  with  the  interface  on  their  computer. 
Users  of  the  MS-DOS  and  Macintosh  systems  answered  different  sets  of 
questions  in  this  section  to  account  for  the  substantially  different  ways  of 
working  with  each  type  of  interface.  However,  questions  were  developed 
around  the  same  fundamental  practices  used  in  each  type  of  interface.  Care 
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was  taken  to  insure  that  the  questions  used  in  this  section  asked  about 
equivalent  operations  for  each  interface.  The  results  from  this  series  of 
questions  were  used  to  determine  whether  there  were  differences  in  users’ 
ratings  of  user-friendliness  for  each  type  of  system. 

Section  5  of  the  questionnaire  was  used  to  answer  Investigative 
Questions  4  and  6.  Research  by  Butler,  Carey,  Herot,  Diagnostic  Research, 
and  Whiteside  was  used  in  developing  this  area  of  research  (3:85,  5:H, 
12:83,  19:1,  28:185).  This  section  also  presented  the  respondent  with  a 
seven-point  Likert  scale.  Users  were  asked  to  use  this  scale  to  respond  to  a 
series  of  statements  developed  by  the  researcher.  In  this  section,  users 
were  given  statements  designed  to  measure  their  satisfaction  with  their 
current  computer  system  and  its  impact  on  their  job.  This  portion  was  used 
to  determine  whether  one  set  of  computer  users  had  a  higher  level  of  user 
satisfaction  than  another  group..  In  addition,  this  portion  was  used  to 
determine  whether  one  population  of  users  felt  that  their  computer  helped 
them  perform  their  job  better  than  the  other  population  of  computer  users. 

Section  6  was  used  to  answer  Investigative  Question  7.  This  section 
was  developed  based  on  the  researcher’s  preliminary  interviews  with 
computer  systems  personnel  in  the  organizations  surveyed.  Respondents 
were  asked  if  they  supervised  other  personnel  or  whether  they  were 
responsible  for  output  generated  by  a  computer.  Users  who  responded 
affirmatively  to  either  question  were  defined  as  responsible  authorities  and 
were  then  asked  to  respond  to  a  series  of  questions  about  the  output  from 
their  computer  system  using  a  seven-point  Likert  scale.  The  responsible 
authorities  were  asked  to  rate  the  quality  of  text,  quality  of  graphics,  and 
the  overall  appearance  of  documents  produced  by  their  computer  system. 
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The  results  from  these  areas  were  used  to  determine  if  there  were  any 
differences  in  the  two  systems  studied  in  quality  of  output. 

Section  7  was  a  set  of  open-ended  questions  about  what  specific 
applications  were  used  in  each  of  the  five  areas  (word  processing, 
spreadsheet,  database,  graphics/presentation,  and  engineering)  studied  for 
Investigative  Question  2.  Users  were  asked  to  write  in  the  name  of  the 
application  used  for  each  area  on  their  computer  system.  The  results  from 
this  section  were  used  to  gather  information  about  what  applications  were 
actually  being  used  to  insure  that  similar-quality  products  were  being  used 
and  that  valid  comparisons  could  be  made  about  the  interface’s  impact  on 
ease-of-use  for  each  type  of  application  studied.  Section  7  was  developed 
as  a  separate  section  at  the  end  of  the  questionnaire  so  that  respondents 
could  simply  write  the  name  of  the  appropriate  programs  on  the 
questionnaire  without  having  to  respond  to  a  multiple  choice  format  on  the 
computerized  answer  sheet.  By  placing  this  section  at  the  end,  separate 
from  Section  3,  the  researcher  felt  that  users  would  not  be  as  likely  to  get 
confused  by  changing  from  computerized  answer  sheets  to  the  actual 
questionnaire.  This  approach  also  simplified  the  data  analysis  process  by 
segregating  the  central  issues  of  the  study  in  Section  3  from  the  peripheral 
control  issues  examined  in  Section  7. 

The  questionnaire  was  submitted  to  the  Commander,  Aeronautical 
Systems  Division,  for  approval  before  sending  to  sampled  organizations. 
Included  in  the  package  submitted  to  his  office  was  a  letter  for  the  Chief  of 
Staff’s  signature  to  sampled  agencies  indicating  his  support  for  the 
research.  Following  his  approval,  the  researcher  assembled  the  packages 
sent  to  each  population.  Packages  were  assembled  in  Air  Force  envelopes 
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and  included  the  cover  letter,  questionnaire  instructions,  survey 
questionnaire,  answer  sheet  (AFIT  Form  1 1C)  and  a  return  Air  Force 
envelope.  The  return  envelope  was  stamped  with  the  researcher’s 
organizational  address  to  facilitate  prompt  return  of  the  questionnaire  and 
improve  the  response  rate.  A  total  of  700  surveys  were  mailed  to  the 
organizations  included  in  the  sample. 

Data  Collection  and  Analysis 

Four  weeks  were  initially  allowed  for  the  return  of  the  questionnaires. 
The  researcher  felt  that  this  period  gave  respondents  ample  time  to  respond 
to  the  survey.  However,  due  to  a  distribution  problem  at  one  of  the 
sampled  agencies,  the  suspense  date  was  extended  for  one  week  to  insure 
that  all  users  had  the  opportunity  to  participate  in  the  survey.  As 
questionnaire  answer  sheets  were  returned,  the  researcher  examined  the 
responses  to  the  demographic  questions  to  insure  that  the  response  met  the 
conditions  established  for  the  sample.  In  addition,  the  researcher  tallied  the 
hand-written  responses  on  the  last  section  of  the  questionnaire  for  those 
respondents  meeting  the  qualifications.  Valid  answer  sheets  were  then 
separated  and  stored  for  optical  scanning.  Answer  sheets  not  meeting  the 
established  conditions  were  separated  and  stored  for  future  reference.  The 
data  was  analyzed  using  the  StatView  statistical  software  package  from 
Brainpower,  Inc.  The  responses  consisted  of  nominal  and  interval  level 
data. 

The  seven-point  Likert  scale  responses  were  assumed  to  be  at  least 
interval  level  data.  In  addition,  because  a  return  rate  of  less  than  100 
percent  was  anticipated,  it  was  assumed  that  the  Central  Limit  Theorem 
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applied  to  this  research.  The  Central  Limit  Theorem  states  that  for  large 
sample  sizes,  30  or  more  cases,  the  data  can  be  assumed  to  be  normally 
distributed. 

It  is  important  to  note  that  this  thesis  is  designed  to  examine  the  user 
interface  only,  and  should  not  be  viewed  as  a  recommendation  for  a 
particular  computer  system.  For  example,  cost  and  connectivity 
considerations  were  not  considered  in  this  evaluation.  While  those 
functions  are  critical  concerns  in  choosing  a  system,  they  were  not 
addressed  in  this  thesis  as  they  are  not  directly  related  to  the  design  of 
computer  interfaces  and  their  effect  on  computer  users. 

In  interpreting  the  data,  it  is  important  to  remember  that  results  were 
obtained  based  on  users'  perceptions,  rather  than  on  actual  observed 
preference  or  usage  patterns.  Questionnaire  responses  were  assumed  to  be 
accurate  and  represent  the  honest  opinions  of  those  surveyed. 

Statistical  Tests 

The  primary  statistical  tests  used  to  analyze  the  data  were  contingency 
table  analysis  and  the  t-test.  Contingency  table  analysis  was  used  to 
determine  whether  two  variables  were  independent.  To  test  the  hypothesis 
that  two  variables  were  independent,  the  researcher  compared  expected 
values  to  observed  values  for  each  variable.  This  result  was  used  to 
determine  the  probability  that  differences  of  the  magnitude  demonstrated 
would  occur  if  each  value  were  actually  independent.  Analyses  which 
rejected  the  null  hypothesis  were  supplemented  by  using  the  t-test  to 
further  examine  the  nature  of  the  relationship  between  the  two  variables  in 
question. 
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The  t-test  was  used  to  determine  the  significance  of  the  difference  of 
means  between  two  populations.  To  test  the  hypothesis  that  the  population 
means  were  equal,  the  researcher  calculated  the  t-statistic  by  dividing  the 
difference  between  the  sample  means  by  the  sum  of  the  sample  standard 
deviations.  The  value  yielded  by  this  result  was  used  to  determine  the 
probability  that  a  difference  of  the  magnitude  demonstrated  would  occur  if 
the  population  means  were  actually  equal.  Differences  were  declared 
significant  at  the  .05  significance  level. 

The  results  of  all  contingency  tables  and  t-tests  were  then  analyzed  to 
determine  the  advantages  and  disadvantages  offered  by  graphical  systems 
as  compared  to  character- based  systems.  Contingency  table  analyses 
rejecting  the  hypothesis  of  independence  followed  by  t-test  results 
indicating  a  significant  difference  between  the  means  for  each  system  were 
presumed  to  indicate  an  advantage  for  one  system  over  its  counterpart.  On 
the  other  hand,  if  the  t-test  did  not  indicate  a  significant  difference  in  the 
two  systems,  it  was  not  listed  as  either  an  advantage  or  disadvantage.  The 
results  of  all  contingency  table  analyses  and  t-tests  were  then  used  to  draw 
conclusions  about  the  design  of  computer  interfaces  and  their  impact  on 
human  effectiveness  and  efficiency  within  Air  Force  organizations. 
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IV.  Data  Analysis 


This  chapter  presents  the  data  analysis  from  the  survey  instrument.  It 
first  outlines  the  demographic  data  for  the  sample  population  followed  by 
the  data  relative  to  each  investigative  question. 

As  stated  in  the  methodology  portion,  a  total  of  700  surveys  were  mailed 
out.  200  of  these  surveys  went  to  the  Advanced  Tactical  Fighter  SPO,  and 
500  of  the  surveys  went  to  the  Wright  Research  and  Development  Center. 
These  figures  reflect  the  difference  in  the  number  of  computer  users  for 
each  organization.  454  of  the  original  700  surveys  were  returned  for  a 
response  rate  of  64.9%.  From  the  454  surveys  returned,  77  were  eliminated 
because  they  failed  to  meet  the  conditions  specified  for  this  research, 
leaving  a  total  of  377  responses  for  analysis.  The  elimination  rate  was 
proportionally  divided  among  the  two  organizations  surveyed.  The  figures 
presented  in  this  chapter  reflect  only  those  377  qualified  responses.  Note 
that  when  percentages  are  used,  results  are  rounded  to  the  nearest  tenth 
and  therefore  may  not  sum  to  exactly  100.0%. 

Demographic  Information 

Question  1  asked  respondents  to  identify  their  organization.  The 
organizational  breakdown  is  presented  in  Table  1  on  the  following  page. 
There  were  no  major  differences  in  the  demographic  responses  for  each 
organization.  Information  presented  in  the  following  tables  is  aggregated 
across  both  organizations. 

Question  2  asked  respondents  which  category  best  described  their 
primary  job.  The  job  categories  and  their  associated  responses  are 
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presented  in  Table  2  on  the  following  page.  The  large  number  of  engineers 
responding  is  indicative  of  the  primary  mission  of  each  of  the  organizations 
included  in  this  study. 


Table  1 

Number  and  Percentage  of  Responses  by  Organization 


Organization 

Number 

Percentage 

Advanced  Tactical  Fighter  SPO 

126 

33.4 

Wright  Research  and  Development  Center 

251 

66.6 

Table  2 

Number  and  Percentage  of  Responses  by  Job  Category 


Number 

Percentage 

Secretarial 

30 

8.0 

Administrative 

19 

5.1 

Management 

89 

23.7 

Engineering 

185 

49.2 

Finance/Cost 

14 

3.7 

Computer  Operations 

25 

6.6 

Security 

1 

.3 

Other 

13 

3.5 

Question  3  asked  respondents  for  their  grade.  The  breakdown  by  grade 
is  presented  in  Table  3  on  the  following  page.  The  few  number  of  enlisted 
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responses  as  well  as  the  large  number  of  senior  civilian  responses  is  again 
indicative  of  the  missions  performed  by  the  organizations  studied. 

Table  3 

Number  and  Percentage  of  Responses  by  Grade 


Grade 

Number 

Percentage 

E1-E3 

0 

0 

E4-E6 

7 

1.9 

E7-E9 

3 

.8 

01-03 

78 

20.9 

04-05 

32 

86 

06  or  above 

9 

2.4 

GS1-GS7 

37 

9.9 

GS8  or  above 

208 

55.6 

Question  4  asked  respondents  the  total  length  of  time  that  they  have 
been  employed  by  the  federal  government.  The  length  of  federal  service 
responses  are  listed  in  Table  4  on  the  following  page.  As  indicated  in  the 
table,  the  sample  used  for  this  research  had  a  high  level  of  experience,  with 
the  largest  group  of  respondents  reporting  over  12  years  of  federal  service. 

Question  5  asked  respondents  how  long  they  have  been  in  their  present 
job.  While  those  with  over  4  years  of  experience  in  their  present  job 
comprised  the  largest  group,  responses  were  well-distributed  throughout 
the  range  of  responses  used  on  the  survey.  The  breakdown  by  length  of 
time  in  present  job  is  presented  in  Table  5  on  the  following  page. 
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Table  4 

Number  and  Percentage  of  Responses  by  Length  of  Federal  Service 


Length  of  Federal  Government  Service 

Number 

Percentage 

Less  than  1  year 

17 

4.6 

At  least  1  but  less  than  3  years 

36 

9.7 

At  least  3  but  less  than  5  years 

53 

14.2 

At  least  5  but  less  than  8  years 

54 

14.5 

At  least  8  but  less  than  12  years 

66 

17.7 

1 2  years  or  more 

147 

39.4 

Table  5 

Number  and  Percentage  of  Responses  by  Length  of  Time  in  Present  Job 


Length  of  Time  in  Present  Job 

Number 

Percentage 

Less  than  1  year 

80 

21.3 

At  least  1  but  less  than  2  years 

79 

21.0 

At  least  2  but  less  than  3  years 

58 

15.4 

At  least  3  but  less  than  4  years 

55 

14.6 

4  years  or  more 

104 

27.7 

Question  6  asked  respondents  for  their  highest  level  of  education.  The 
number  and  percentage  responses  by  education  level  are  presented  in  Table 
6  on  the  following  page.  The  relatively  high  education  level  again  reflects 
the  job  and  grade  structure  presented  earlier. 


Table  6 

Number  and  Percentage  of  Responses  by  Education  Level 


Highest  Level  of  Education 

Number 

Percentage 

High  school  graduate 

17 

4.5 

Some  college 

47 

12.5 

Associate's  degree 

9 

2.4 

Bachelor’s  degree 

164 

43.6 

Master’s  degree 

122 

32.4 

Doctoral  degree 

17 

4.5 

Question  7  asked  respondents  whether  they  used  a  computer  on  the  job 
and  was  only  used  as  a  means  to  screen  responses  for  Questions  8-11. 

Those  users  responding  that  they  did  not  use  a  computer  on  the  job  were 
eliminated  from  consideration  and  were  not  included  in  any  of  the  analyses 
used  in  this  study. 

Of  those  respondents  indicating  that  they  used  a  computer  on  the  job  in 
Question  7.  Question  8  asked  respondents  what  type  of  computer  they  used 
most  often.  To  control  the  type  of  text-based  and  graphical  interfaces  used 
in  this  analysis,  only  respondents  using  IBM  compatible  or  Apple  Macintosh 
computers  were  included  in  this  study.  MS-DOS  compatible  systems  made 
up  the  largest  group  of  users;  however,  the  number  of  Macintosh  users  was 
also  substantial.  The  large  number  of  respondents  in  each  category  allowed 
the  researcher  to  conduct  a  thorough  analysis  for  each  of  the  variables 
studied.  Breakdown  of  responses  by  type  of  computer  used  most  often  on 
the  job  is  presented  in  Table  7  on  the  following  page. 
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Table  7 

Number  and  Percentage  of  Responses  by  Computer  Type 


Type  of  computer  used  most  often 

Number 

Percentage 

Zenith  Z-248  or  other  IBM  compatible 

218 

57.8 

Apple  Macintosh 

159 

42.2 

Other* 

0 

0 

*  Respondents  choosing  “other"  were  disqualified  under  the  parameters 

established  for  this  study  and  were  not  considered  in  the  data  analysis. 

Question  9  asked  respondents  how  long  they  have  used  the  computer 
specified  in  their  response  to  Question  8.  In  order  to  ensure  that 
respondents  were  able  to  conduct  a  fair  evaluation  of  the  interfaces 
examined  in  thi'  study,  users  with  less  than  three  months’  experience  were 
eliminated  and  were  not  included  in  the  analysis. 

As  shown  in  the  table,  the  largest  group  of  respondents  reported  over 
one  year  of  experience  on  the  computer  system  used  on  the  job.  274  of  the 
377  total  users  (72.7%)  were  in  the  most  experienced  category.  103  of  the 
377  users  responding,  or  27.3%,  reported  using  their  system  between  three 
months  and  one  year.  The  researcher  felt  that  this  large  base  of 
experienced  users  ensured  that  respondents  were  adequately  qualified  to 
evaluate  each  interface  thoroughly  and  enhanced  the  validity  of  the  study. 
Table  8  on  the  following  page  presents  the  number  and  percentage  of 
responses  for  length  of  computer  use. 
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Table  8 

Number  and  Percentage  of  Responses  by  Length  of  Use  for  Computer 
Specified  in  Response  to  Question  8 


Length  of  Use 

Number 

Percentage 

Less  than  3  months* 

0 

0 

At  least  3  months  but  less  than  1  year 

103 

27.3 

1  year  or  more 

274 

72.7 

*  Respondents  choosing  “less  than  3  months"  were  disqualified  under  the 
parameters  established  for  this  study  and  were  not  considered  in  the  data 
analysis. _ _ _ _ 

Question  10  asked  users  approximately  how  many  hours  per  week  they 
use  their  computer  on  the  job.  As  described  in  Chapter  III,  respondents  who 
used  their  computer  less  than  three  hours  per  week  were  disqualified  from 
further  data  analysis  under  the  parameters  established  for  this  study.  Table 
9  on  the  following  page  presents  the  number  and  percentage  of  responses 
for  each  category. 

Question  1 1  asked  respondents  how  many  different  software  packages 
they  use  on  the  job.  As  described  earlier  in  the  study,  only  those 
respondents  using  more  than  one  software  package  on  the  job  were 
considered  in  the  data  analysis  under  the  parameters  established  for  this 
research.  Breakdown  by  numbers  of  software  packages  used  is  presented 
In  Table  10  on  the  following  page. 
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Table  9 

Number  and  Percentage  of  Responses  by  Hours  of  Computer  Use  Per  Week 


Hours  of  computer  use  per  week 

Number  Percentage 

Less  than  3* 

0 

0 

At  least  3  but  less  than  5 

45 

12.0 

At  least  5  but  less  than  10 

87 

23.1 

At  least  10  but  less  than  20 

145 

38.6 

20  or  more 

98 

26.1 

*  Respondents  choosing  “less  than  3" 

were  disqualified  under  the 

parameters  established  for  this  study  and  were  not  considered  in  the  data 

analysis. 

Table  10 

Number  and  Percentage  of  Responses  by  Number  of  Software  Packages 

Used  on  the  Job 


Number  of  Software  Packages  Used 

Number 

Percentage 

1* 

0 

0 

2-3 

100 

26.5 

4-6 

149 

39.5 

Over  6 

128 

34.0 

*  Respondents  answering  “1"  were  disqualified  under  the  parameters 

established  for  this  study  and  were  not  considered  in  the  data  analysis. 

Question  17  asked  respondents  to  rate  their  computer  skill  level  on  a 
scale  of  1  to  5,  with  1  being  the  least  advanced  and  5  being  the  most 
advanced.  The  overall  number  and  percentage  of  responses  are  reported 
for  demographic  purposes  in  Table  1 1  below. 

Table  1 1 

Number  and  Percentage  of  Responses  by  User  Computer  Skill  Level 


User  Computer  Skill  Level 

Number 

Percentage 

1 

8 

2.2 

2 

26 

7.1 

3 

157 

43.1 

4 

115 

31.6 

5 

58 

15.9 

Investigative  Question  1 

The  researcher  used  contingency  table  analysis  to  test  the  hypothesis 
that  type  of  computer  used  on  the  job  and  time  required  to  learn  the  basic 
system  were  independent.  Contingency  table  analysis  calculates  an 
expected  number  of  responses,  based  on  the  number  of  responses  for  each 
group,  and  compares  that  figure  with  the  actual  observed  number  of 
responses  for  each  group.  In  all  of  the  contingency  tables  presented  in  this 
study,  users  responding  that  they  did  not  use  that  application  were 
eliminated  from  the  analysis.  These  observed  and  expected  frequencies  are 
presented  in  Table  12  below.  Observed  frequencies  are  listed  first, 
followed  by  expected  frequencies,  indicated  by  parentheses. 
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As  shown  in  the  table,  the  greatest  discrepancies  between  observed  and 
expected  frequencies  exist  at  the  extreme  ends  of  the  response  scale  while 
the  middle  range  frequencies  do  not  deviate  markedly  from  the  expected 
values. 


Table  12 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Time  Required  to  Learn  the  Basic  System 


Hours 

MS-DOS  users 

Macintosh  users 

Totals 

Less  than  1 

49 

59 

108 

(61.14) 

(46.86) 

1-3 

70 

54 

124 

(70.2) 

(53.8) 

3-5 

34 

21 

55 

(31.14) 

(23.86) 

5-7 

7 

7 

24 

(13.59) 

(10.41) 

More  than  7 

27 

10 

37 

(20.95) 

(16.05) 

Totals 

197 

151 

348 

Total  Chi-Square  -  12.173 

P-.0161 

46 


The  researcher  was  forced  to  combine  response  6  with  response  5  to 
meet  the  minimum  requirement  of  5  responses  per  block  for  contingency 
table  analysis.  As  shown  in  the  table,  the  overall  p  value  for  the  Chi-Square 
value  obtained  for  this  analysis  was  .0161.  This  p  value  strongly  suggests 
that  type  of  computer  used  on  the  job  and  time  required  to  learn  the  basic 
system  are  not  independent. 

Based  on  the  results  obtained  in  the  contingency  table  analysis,  the 
researcher  conducted  a  one-tail  t-test  to  determine  whether  the  mean 
response  for  users  of  teit-based  interfaces  was  higher  than  that  for  users  of 
graphical  user  interfaces  (a  higher  mean  indicating  that  time  required  to 
learn  the  basic  system  was  higher).  Again,  for  all  t-tests  used  in  this  study, 
users  responding  that  they  did  not  use  that  particular  application  were 
eliminated  from  the  analysis.  The  results  from  the  t-test  are  presented  in 
Table  13  on  the  following  page. 

The  results  of  the  t-test  suggest  a  significant  difference  in  the  mean 

% 

time  required  to  learn  the  basic  system  for  each  group.  The  results  of  the  t- 
test  were  significant  at  the  .0006  level.  These  results  are  consistent  with 
the  relationship  suggested  by  the  contingency  table  analysis  in  Table  12. 

Viewing  the  contingency  table  analysis  and  t-tests  together  strongly 
suggests  that  there  is  a  dependent  relationship  between  time  required  to 
learn  the  basic  system  and  type  of  computer  used.  In  addition,  one  may 
conclude  that  users  of  graphical  user  interfaces  in  the  organizations 
surveyed  require  significantly  less  time  to  learn  the  basic  system  than  their 
counterparts  using  text-based  user  interfaces. 
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Table  13 

T-Test  Results  for  Type  of  Computer  vs.  Time  Required  to 
Learn  the  Basic  System 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

197 

2.594 

1.504 

Macintosh  users 

151 

2.093 

1.298 

t  value  =  3.267 

p  -  .0006 

Note:  Mean  indicates  response  number,  not  hours 

Investigative  Question  2 

To  answer  Investigative  Question  2,  the  researcher  used  contingency 
table  analysis  and  t-test  comparisons  for  the  average  amount  of  time  it  took 
respondents  to  learn  a  new  application  program.  As  a  supplement,  the 
researcher  also  conducted  similar  analyses  for  specific  types  of  applications 
to  determine  if  learning  time  for  each  type  of  interface  was  task -dependent. 
The  specific  types  of  applications  examined  were  word  processing, 
spreadsheet,  database,  graphics/presentation,  and  CAD/CAM.  Based  on 
preliminary  interviews  with  representatives  at  each  of  the  organizations 
used  in  this  research,  the  researcher  found  that  these  applications  were 
common  to  each  organization.  The  researcher  also  felt  that  these 
application  areas  offered  the  best  opportunity  to  evaluate  a  wide  range  of 
tasks  that  are  common  in  most  of  today’s  office  environments. 
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To  test  whether  time  required  to  learn  word  processing  was 
independent  of  type  of  interface  used,  the  researcher  used  contingency 
table  analysis.  Table  14  below  presents  the  observed  and  expected 
frequencies  following  the  conventions  presented  for  Investigative 
Question  1. 


Table  14 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Time  Required  to  Learn  Primary  Word  Processing  Application 


Hours 

MS-DOS  users 

Macintosh  users 

Totals 

Less  than  4 

80 

99 

179 

(102.56) 

(76.44) 

4-8 

53 

40 

93 

(53.29) 

(39.71) 

8-12 

24 

7 

31 

(17.76) 

(13.24) 

12-16 

10 

5 

15 

(8.59) 

(6.41) 

More  than  16 

45 

7 

52 

(29.79) 

(22.21) 

Totals 

212 

158 

370 

Total  Chi-Square  -  35.467 

p  -  .0001 _ 


Once  again,  the  greatest  discrepancies  between  observed  and  expected 
frequencies  occur  at  the  extreme  ends  of  the  response  scale.  For  Macintosh 
users,  there  were  more  responses  than  expected  below  the  4  hour  point, 
while  there  were  fewer  responses  than  expected  above  the  8  hour  mark. 
MS-DOS  responses  were  opposite  of  those  found  for  Macintosh  users 
reported  above. 

Based  on  the  low  p  value  associated  with  the  contingency  table,  the 
researcher  followed  this  analysis  with  a  one-tail  t-test  to  determine  if  the 
mean  time  required  for  command-based  interface  users  to  learn  their 
primary  word  processing  application  was  statistically  different  from  the 
mean  time  for  graphical  user  interface  users.  Table  15  below  presents  the 
results  of  the  t-test. 


Table  15 

T-Test  Results  for  Type  of  Computer  vs.  Time  Required  to 
Learn  Primary  Word  Processing  Application 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

212 

2.684 

1.964 

Macintosh  users 

158 

1.665 

1.224 

t  value  »  5.745 

p  =  .0001 

Note;  Mean  indicates  response  number,  not  hours 
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The  t-test  clearly  shows  a  difference  in  the  means  for  each  group, 
indicating  that  MS-DOS  users  feel  that  it  takes  them  longer  to  learn  their 
primary  word  processing  program  than  Macintosh  users  do.  These  results 
coupled  with  the  contingency  table  analysis  suggest  that  there  may  be  a 
dependent  relationship  between  type  of  computer  used  and  time  required  to 
learn  word  processing  applications. 

The  researcher  then  conducted  similar  analyses  for  spreadsheet 
applications.  As  in  previous  tables,  the  greatest  differences  between 
observed  and  expected  values  for  this  variable  occurred  at  the  extreme 
points  of  the  rating  scale.  For  example,  only  53  MS-DOS  users  reported  that 
it  took  them  less  than  4  hours  to  learn  their  primary  spreadsheet  program 
while  65.5  responses  were  expected.  On  the  other  hand,  68  Macintosh  users 
were  in  the  lowest  category  with  only  55.5  responses  expected.  Conversely, 
at  the  top  end  of  the  scale.  28  MS-DOS  users  reported  taking  over  12  hours 
to  learn  their  primary  spreadsheet  program  with  19.49  responses  expected 
in  this  category.  Only  8  Macintosh  users  reported  taking  over  12  hours  to 
learn  their  primary  spreadsheet  with  16.51  responses  expected.  Observed 
responses  for  the  categories  over  4  hours  and  less  than  12  hours  were  more 
closely  aligned  with  the  expected  number  of  responses  in  each  respective 
category. 

The  results  of  the  contingency  table  analysis  are  presented  in  Table  16 
on  the  following  page.  Again,  it  was  necessary  to  combine  frequencies  for 
responses  4-7  to  meet  the  5  observed  responses  per  cell  constraint  needed 
for  contingency  table  analysis. 
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Table  16 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Time  Required  to  Learn  Primary  Spreadsheet  Application 


Hours 

MS-DOS  users 

Macintosh  users 

Totals 

Less  than  4 

53 

68 

121 

(65.5) 

(55.5) 

4-8 

36 

24 

60 

(32.48) 

(27.52) 

8-12 

14 

11 

25 

(13.53) 

(11.47) 

More  than  1 2 

28 

8 

36 

(19.49) 

(16.51) 

Totals 

131 

111 

242 

Total  Chi-Square  =  14.175 

p  =  .0027 

Due  to  the  low  p  value,  the  researcher  was  able  to  conclude  that 
computer  type  and  time  required  to  learn  the  user’s  primary  spreadsheet 
program  were  not  independent.  The  researcher  then  conducted  t-tests  to 
determine  whether  there  was  a  difference  in  the  mean  time  required  to 
learn  the  user’s  primary  spreadsheet  program.  Results  of  the  t-test  are 
shown  in  Table  17. 
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Table  17 

T-Test  Results  for  Type  of  Computer  vs.  Time  Required  to 
Learn  Primary  Spreadsheet  Application 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

131 

2.427 

1.745 

Macintosh  users 

111 

1.73 

1.265 

t  value  =  3.504 

p  -  .0002 

Note:  Mean  indicates  response  number,  not  hours 

The  results  of  the  t-test  were  consistent  with  what  one  might  expect 
after  viewing  the  contingency  table.  The  researcher  concluded  that  time 
required  to  learn  the  user’s  primary  spreadsheet  program  was  dependent  on 
the  type  of  computer  used.  In  addition,  the  researcher  concluded  that  users 
of  graphical  user  interfaces  feel  that  they  require  less  time  to  learn 
spreadsheet  applications  than  their  counterparts  using  text-based 
interfaces. 

As  with  word  processing  and  spreadsheet  applications,  the  researcher 
examined  database  applications  using  a  contingency  table.  For  the 
contingency  table,  responses  4-7  were  combined  with  response  3  in  order 
to  meet  the  required  5  frequencies  per  cell.  Required  and  expected 
frequencies  are  presented  in  Table  18  on  the  following  page. 
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Table  18 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Time  Required  to  Learn  Primary  Database  Application 


Hours 

MS-DOS  users 

Macintosh  users 

Totals 

Less  than  5 

26 

19 

45 

(32.73) 

(12.27) 

5-10 

27 

11 

38 

(27.64) 

(10.36) 

More  than  10 

43 

6 

49 

(35.64) 

(13.36) 

Totals 

96 

36 

132 

Total  Chi-Square  *  10.703 

p  » .0047 

Like  tables  for  word  processing  and  spreadsheet  applications,  the 
contingency  table  analysis  for  database  established  meaningful  differences 
between  observed  and  expected  frequencies.  Based  on  the  low  p  value,  the 
researcher  was  able  to  reject  the  hypothesis  that  time  required  to  learn 
database  programs  and  type  of  computer  used  were  independent. 

Because  there  was  evidence  to  indicate  possible  dependency  between 
the  two  variables,  a  one-tail  t-test  was  run  to  detect  differences  between 
the  mean  responses  for  each  group  of  users.  Table  19  presents  the  results 
of  this  t-test. 
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Table  19 

T-Test  Results  for  Type  of  Computer  vs.  Time  Required  to 
Learn  Primary  Database  Program 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

96 

3.094 

2.083 

Macintosh  users 

36 

1.639 

.762 

t  value  -  3.504 

p  -  .0002 

Note;  Mean  indicates  response  number,  not  hours 

The  results  of  the  t-test  were  consistent  with  the  finding  for  word 
processing  and  spreadsheet  applications  indicating  a  significant  difference 
in  the  amount  of  time  required  to  lea  n  user’s  primary  database  application. 

The  researcher  concluded  that  time  required  to  learn  the  user’s  primary 
database  application  was  dependent  on  the  type  of  computer  used.  In 
addition,  the  researcher  concluded  that  users  of  graphical  user  interfaces 
feel  that  they  require  less  time  to  learn  database  applications  than  their 
counterparts  using  text-based  interfaces. 

The  researcher  then  conducted  the  contingency  table  analysis  for 
graphics/presentation  programs  using  Table  20  on  the  following  page.  As 
with  the  tables  presented  earlier  in  the  chapter,  responses  5-7  were  merged 
with  response  4  to  yield  the  required  number  of  responses  for  contingency 
table  analysis  in  each  cell. 
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Table  20 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Time  Required  to  Learn  Primary  Graphics/Presentation  Application 


Hours 

MS-DOS  users 

Macintosh  users 

Totals 

Less  than  4 

55 

73 

128 

(70.1) 

(57.9) 

4-8 

51 

52 

103 

(56.4) 

(46.6) 

8-12 

28 

15 

43 

(23.55) 

(19.45) 

More  than  12 

50 

12 

62 

(33.95) 

(28.05) 

Totals 

184 

152 

336 

i 

Total  Chi-Square  =  26.958 

p  =  .0001 

Consistent  with  the  results  for  each  of  the  preceding  applications,  there 
are  noticeable  differences  between  actual  and  expected  values  for 
graphics/presentation  programs.  For  times  below  8  hours,  text-based 
interface  users  have  fewer  responses  than  expected,  while  graphical 
interface  users  have  more  responses  than  expected.  This  situation  is 
reversed  for  responses  above  8  hours. 

Based  on  the  significance  level  (indicated  by  the  p  value)  the  researcher 
was  able  to  reject  the  hypothesis  that  time  requr^  d  to  learn 
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graphics/presentation  programs  and  type  of  computer  system  used  are 
independent.  Because  the  researcher  was  able  to  reject  the  hypothesis  of 
independence,  he  then  conducted  the  t-test  to  further  examine  the  nature  of 
the  relationship  between  the  two  variables.  Table  21  presents  the  results  of 
the  one-tail  t-test. 


Table  21 

T-Test  Results  for  Type  of  Computer  vs.  Time  Required  to 
Learn  Primary  Graphics/Presentation  Application 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

184 

2.772 

1.838 

Macintosh  users 

152 

1.888 

1.274 

t  value  =  5.014 

p  -  .0001 

Note:  Mean  indicates  response  number,  not  hours 

The  results  of  the  t-test  show  a  clear  difference  in  the  means  for  MS- 
DOS  and  Macintosh  users.  The  significance  of  the  t-test  in  combination 
with  the  significance  of  the  contingency  table  test  for  independence 
suggests  that  Macintosh  users  take  less  time  to  learn  graphics/presentation 
programs  than  their  MS-DOS  user  counterparts. 

The  final  application-specific  area  that  was  examined  was  CAD/CAM 
software.  Again,  the  researcher  used  a  contingency  table  to  test  whether 
time  required  to  learn  CAD/CAM  software  is  independent  of  type  of 
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computer  system  used.  The  observed  and  expected  frequencies  are 
presented  in  Table  22.  Here,  responses  1  and  2  were  combined  as  were 
responses  3-7  in  order  to  yield  the  required  5  responses  per  cell  needed  for 
contingency  table  analysis. 

Table  22 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 

Time  Required  to  Learn  Primary  CAD/CAM  Application 


Hours 

MS-DOS  users 

Macintosh  users 

Totals 

0-10 

19 

9 

28 

(21.12) 

(6.88) 

More  than  1 0 

24 

5 

29 

(21.88) 

(7.12) 

Totals 

43 

14 

57 

Total  Chi-Square  =  1.707 

p  =  .1913 

In  contrast  to  the  other  applications  examined,  the  researcher  was  not 
able  to  reject  the  hypothesis  that  time  required  to  learn  CAD/CAM  software 
and  type  of  computer  system  used  are  independent.  Because  potential 
dependency  between  the  two  variables  could  not  be  established,  no  t-test 
was  conducted  as  any  difference  in  the  means  would  hold  little  practical 
meaning. 
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The  final  question  respondents  answered  that  was  related  to 
Investigative  Question  2  asked  computer  users  on  the  average  how  long  it 
took  them  to  learn  a  new  application  program.  As  in  the  application- 
specific  questions,  responses  4-7  had  to  be  combined  to  yield  at  least  5 
observed  responses  in  each  cell.  Table  23  shows  the  observed  and  expected 
frequencies  for  average  time  required  to  learn  a  new  application  and  the 
type  of  computer  system  used  on  the  job. 

Table  23 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 

Average  Time  Required  to  Learn  a  New  Application 


Hours 

MS-DOS  users 

Macintosh  users 

Totals 

Less  than  4 

71 

90 

161 

(93.05) 

(67.95) 

4-8 

67 

47 

114 

(65.89) 

(48.11) 

8-12 

31 

11 

42 

(24.27) 

(17.73) 

More  than  12 

46 

9 

55 

(31.79) 

(23.21) 

Totals 

215 

157 

372 

Total  Chi-Square  -  31.898 

p  -  .0001 
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The  greatest  differences  between  observed  and  expected  values  was  at 
the  extreme  ends  of  the  response  scale.  The  table  shows  that  there  were 
fewer  responses  below  4  hours  for  MS-DOS  users  than  expected,  while 
Macintosh  responses  were  much  higher  than  expected.  This  situation  is 
reversed  above  8  hours  as  MS-DOS  users  had  higher  observed  frequencies 
than  expected  and  Macintosh  users  had  fewer  responses  in  these  categories 
than  expected.  This  situation  repeats  a  pattern  observed  in  most  of  the 
application-specific  questions. 

The  p  value  of  .000 1  allowed  the  researcher  to  reject  the  hypothesis 
that  average  time  required  to  learn  a  new  application  and  type  of  computer 
system  used  are  independent.  Because  a  majority  of  the  application- 
specific  areas  also  rejected  the  hypothesis  of  independence,  one  would 
expect  a  similar  result  for  the  more  general  question  examined  here. 

Based  on  this  evidence,  the  researcher  then  conducted  the  one-tail  t-test 
to  test  whether  the  means  for  one  user  group  was  significantly  higher  than 
that  of  the  other  user  group.  The  results  of  the  t-test,  as  illustrated  in  Table 
24.  indicate  that  the  mean  for  users  of  text-based  interfaces  is  higher  than 
the  mean  for  users  of  graphics-based  interface  systems.  The  extremely  low 
p-value  of  .0001  indicates  a  very  low  probability  that  the  results  indicated 
in  the  table  occurred  by  chance. 

The  results  of  the  t-test  combined  with  the  contingency  table  analysis 
presented  in  Table  23  strongly  suggest  that,  on  the  average,  applications 
employing  graphical  user  interfaces  require  significantly  less  time  to  learn 
than  applications  using  standard  command-driven  interfaces. 
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Table  24 

T-Test  Results  for  Type  of  Computer  vs.  Average  Time 
Required  to  Learn  a  New  Application 


In  conclusion,  there  appears  to  be  strong  evidence  that,  for  most 
applications,  the  amount  of  time  required  to  learn  that  application  is 
dependent  on  the  type  of  interface  used.  For  most  applications,  graphical 
user  interfaces  require  significantly  less  time  to  learn  than  their  text-based 
counterparts. 

In  particular,  word  processing,  spreadsheet,  database,  and 
graphics/presentation  programs  seem  to  lend  themselves  particularly  well 
to  graphics-based  interfaces.  Only  CAD/CAM  programs  appear  to  be 
independent  of  learning  time.  There  was  no  evidence  that  text-based 
interfaces  were  better  for  any  of  the  application  areas  examined  in  this 
study.  On  the  average,  users  of  graphical  user  interfaces  reported  that  they 
required  less  time  to  learn  a  new  application  than  did  users  of  text-based 
systems. 
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Care  should  be  used  in  interpreting  the  results  for  Investigative 
Question  2.  Because  of  the  differences  in  operating  systems  and  software 
availability,  it  was  not  always  possible  to  compare  the  same  application  for 
MS-DOS  users  and  Macintosh  users.  To  overcome  this  potential  limitation, 
the  researcher  asked  respondents  to  provide  the  name  of  the  software 
package  they  used  in  each  of  the  categories  examined.  In  each  case, 
popular  and  highly-rated  commercial  programs  were  the  norm.  For 
example,  in  the  spreadsheet  category  the  majority  of  MS-DOS  users  used 
Lotus  1-2-3,  a  package  unavailable  for  the  Macintosh.  However,  Macintosh 
users  used  the  most  popular  Macintosh  spreadsheet,  Microsoft  Excel,  almost 
exclusively.  Readers  should  be  aware  that  because  it  was  not  always 
possible  to  compare  identical  vendors’  products  for  each  of  the  application 
areas,  extreme  care  must  be  used  in  generalizing  the  results  of  this  portion 
of  the  study  to  all  situations. 

Investigative  Question  3 

To  answer  Investigative  Question  3.  the  researcher  examined  user 
responses  to  two  questions  for  each  interface  type.  For  MS-DOS  users, 
question  46  asked  users  to  rate  the  ease  of  use  of  their  system  for 
performing  basic  file  operations  and  question  47  asked  users  to  rate  the 
overall  ease  of  use  for  their  computer.  Questions  54  and  55  asked  the  same 
questions  respectively  for  Macintosh  users.  Responses  to  these  questions 
were  used  to  determine  the  overall  user  friendliness  for  each  type  of 
interface. 

The  researcher  first  examined  observed  and  expected  frequencies  using 
a  contingency  table  to  test  for  variable  independence.  For  ease  of  use  in 
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performing  basic  file  operations,  0  MS-DOS  and  Macintosh  users  chose 
response  1  (extremely  difficult),  4  MS-DOS  users  and  0  Macintosh  users 
chose  response  2  (difficult),  while  7  MS-DOS  and  2  Macintosh  users  chose 
response  3  (moderately  difficult).  Because  of  the  small  number  of  users  in 
these  categories,  particularly  among  Macintosh  users,  these  responses  were 
not  reported  in  the  table  so  as  to  maintain  the  minimum  of  5  observed 
responses  per  cell.  The  researcher  also  felt  that  by  not  including  these 
responses  in  the  table,  the  reader  was  able  to  get  more  accurate 
representation  of  actual  responses  for  each  group  by  eliminating  the 
distortion  caused  by  combining  categories  over  a  wide  range  that  included 
very  few  responses  on  the  low  end  of  the  range.  The  researcher  did 
combine  responses  4  and  5  (moderate  and  moderately  easy)  to  insure  there 
were  a  minimum  of  5  observed  values  in  each  cell.  Table  25  shows  the 
observed  and  expected  frequencies  for  ease  of  use  in  performing  basic  file 
operations  and  computer  used. 

The  values  in  the  table  show  that  MS-DOS  users  rated  the  ease  of  use 
for  performing  the  basic  file  operations  consistently  lower  than  Macintosh 
users.  For  MS-DOS  users,  frequencies  were  higher  than  expected  for  the 
moderate  to  moderately  easy  range,  while  Macintosh  responses  were  higher 
for  the  easy  and  extremely  easy  response  categories.  The  extremely  low  p- 
value  of  .0001  indicated  for  the  table  allowed  the  researcher  to  reject  the 
hypothesis  that  ease  of  use  for  performing  the  basic  file  operations  and  type 
of  computer  system  used  are  independent. 
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Table  25 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Ease  of  Use  in  Performing  Basic  File  Operations 


MS-DOS  users 

Macintosh  users 

Totals 

Moderate  - 
moderately  easy 

88 

18 

106 

(59.81) 

(46.19) 

Easy 

73 

66 

139 

(78.43) 

^60.57) 

Extremely  easy 

41 

72 

113 

(63.76) 

(49.24) 

Totals 

202 

156 

358 

Total  Chi-Square  -  49.998 

p  -  .0001 

To  further  examine  the  relationship  between  these  two  variables,  the 
researcher  then  performed  a  one-tail  t-test  to  determine  if  the  mean  score 
for  graphical  interface  users  was  higher  than  the  mean  score  for  text-based 
interface  users  on  ease  of  use  in  performing  basic  file  operations.  Table  26 
on  the  following  page  shows  the  results  for  this  t-test.  Note  that  all 
responses  were  considered  in  determining  the  mean  responses  for  each 
group. 
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Table  26 

T-Test  Results  for  Type  of  Computer  vs.  Ease  of  Use  in 
Performing  Basic  File  Operations 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

213 

5.446 

1.175 

Macintosh  users 

158 

6.297 

.786 

t  value  -  -7.892 

p-.OOOl 

The  results  of  the  t-test  confirmed  that  users  of  graphical  user 
interfaces  in  the  organizations  surveyed  rate  ease  of  use  for  performing 
basic  file  operations  significantly  higher  than  their  counterparts  using  text- 
based  interfaces.  When  contingency  table  and  t-test  results  are  considered 
together,  there  appears  to  be  strong  evidence  that  ease  of  use  in  performing 
basic  file  operations  is  not  independent  of  type  of  computer  system  used 
and  that  users  of  graphics-based  systems  rate  ease  of  use  for  basic  file 
operations  higher  than  users  of  text-based  interfaces. 

The  researcher  then  examined  the  contingency  table  for  overall  ease  of 
use  for  each  type  of  computer  system.  Though  not  illustrated  in  the  table 
for  the  reasons  discussed  in  the  previous  analysis.  0  users  in  either  group 
chose  response  1  (extremely  difficult).  3  MS-DOS  users  and  0  Macintosh 
users  chose  response  2  (difficult),  and  4  MS-DOS  users  and  2  Macintosh 
users  chose  response  3  (moderately  difficult).  The  researcher  combined 
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responses  4  and  5  to  insure  there  were  a  minimum  of  5  observed  responses 
per  cell.  Table  27  presents  the  results  for  this  variable. 


Table  27 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 

Overall  Ease  of  Use 


MS-DOS  users 

Macintosh  users 

Totals 

Moderate  - 
moderately  easy 

114 

12 

126 

(70) 

(56) 

Easy 

60 

63 

123 

(68.33) 

(54.67) 

Extremely  easy 

21 

81 

102 

(56.67) 

(45.33) 

Totals 

195 

156 

351 

Total  Chi-Square  =  1 15.025 

p  =  .0001 

The  pattern  in  Table  27  is  identical  to  that  in  Table  25.  MS-DOS  users 
had  higher  than  expected  frequencies  in  the  low  ranges  of  the  response 
scale  while  Macintosh  users  had  higher  than  expected  frequencies  at  the  top 
of  the  scale.  The  low  p  value  in  this  table  again  permitted  the  researcher  to 
reject  the  hypothesis  that  the  two  variables  examined  are  independent. 

The  researcher  then  performed  the  one-tail  t-test  to  test  if  the  mean 
score  for  MS-DOS  users  was  actually  lower  than  the  mean  score  for 
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Macintosh  users.  Table  28  below  presents  the  results  for  the  t-test.  Again 
note  that  all  responses  were  considered  in  calculating  the  mean  scores  and 
are  reported  in  the  table. 


Table  28 

T-Test  Results  for  Type  of  Computer  vs.  Overall  Ease  of  Use 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

202 

5.193 

1.059 

Macintosh  users 

158 

6.392 

.756 

t  value  =  -12.036 

p  =  .0001 

As  in  the  contingency  table,  the  results  of  the  t-test  for  overall  ease  of 
use  parallel  those  for  the  basic  file  operations  shown  in  Table  26.  The  mean 
score  for  text-based  interface  users  for  overall  ease  of  use  was  significantly 
lower  than  the  mean  score  for  graphics-based  interface  users  in  this  study 
(p  =  .0001 ).  Considered  with  the  results  of  the  contingency  table,  one  can 
conclude  that  overall  ease  of  use  is  not  independent  of  computer  system 
type  and  that  users  of  graphical  interface  systems  rate  overall  ease  of  use 
significantly  higher  than  users  of  text-based  interface  systems. 

The  results  from  each  of  these  tables  allowed  the  researcher  to  conclude 
that  graphical  user  interfaces  have  higher  ratings  of  user-friendliness  than 
text-based  interfaces.  For  each  of  the  questions  examined,  statistical  tests 
consistently  showed  graphical  user  interfaces  with  higher  levels  of  user- 
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reported  ease  of  use  ratings.  Significance  levels  were  consistently  at  the 
.000 1  level,  indicating  strong  support  for  the  conclusions  reported. 

This  section  of  the  survey  also  asked  users  to  rate  the  user-friendliness 
of  specific  aspects  of  using  their  computer  system.  Mean  responses  for 
questions  40-45  are  reported  for  MS-DOS  users  in  Table  29  below. 

Table  29 

Mean  Responses  for  MS-DOS  Operations 


Type  of  Operation 

Mean  Response 

Std  Deviation 

Using  software  commands 

5.096 

1.123 

Using  the  keyboard 

5.315 

1.156 

Working  with  the  function  keys 

5.197 

1.200 

Recalling  meanings  of  function  keys 

3.787 

1.333 

Learning  the  meanings  of  function 
keys 

4.258 

1.220 

Moving  the  cursor  to  different 
points  around  the  screen 

5.571 

1.222 

Similarly,  Table  30  on  the  following  page  reports  responses  for 
Macintosh  operations  measured  by  questions  48-53  in  the  Human- 
Computer  Interface  Survey. 

The  researcher  did  not  do  any  further  statistical  analysis  on  these 
operations  due  to  the  difficulty  in  establishing  a  one-to-one  relationship 
between  the  MS-DOS  and  Macintosh  operations.  Attempts  to  compare 
operations  between  the  two  systems  should  be  done  carefully.  Any 
conclusions  from  these  results  are  left  to  the  reader. 


68 


Table  30 

Mean  Responses  for  \*?::intosh  Operations 


Type  of  Operation 

Mean  Response 

Std  Deviation 

Working  with  the  icons 

6.416 

.826 

Understanding  what  icons  represent 

5.994 

1.114 

Working  with  the  Macintosh 
windows 

6.231 

.979 

Operating  the  mouse 

6.563 

.662 

Learning  to  use  the  mouse 

6.561 

.760 

Moving  the  cursor  to  different 
points  around  the  screen 

6.562 

.661 

Investigative  Question  4 

Investigative  Question  4  examined  whether  users  of  one  type  of 
interface  felt  more  strongly  that  their  computer  helps  them  perform  in  their 
job.  The  same  statistical  tests  as  were  used  in  Investigative  Questions  1-3 
were  used  to  answer  Investigative  Question  4. 

Table  31  on  the  following  page  presents  the  observed  and  expected 
values  for  the  perception  that  the  computer  system  used  helps  users 
perform  in  their  job  and  the  type  of  computer  system  actually  used.  Note 
that  0  users  in  each  group  answered  between  the  range  of  Response  1- 
Response  3  (strongly  disagree-slightly  disagree).  Responses  4-5  were 
combined  to  yield  the  required  5  observed  values  per  cell. 

As  illustrated  in  the  table,  observed  responses  on  the  lower  end  of  the 
response  scale  were  in  the  range  that  was  expected.  However,  at  the  top 
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end  of  the  response  scale.  Macintosh  users  chose  the  top  value  more  often 
than  expected  and  MS-DOS  users  were  more  inclined  to  choose  response  6 
over  the  highest  value.  Based  on  this  contingency  table,  the  researcher  was 
again  able  to  reject  the  hypothesis  that  users’  perceptions  that  their 
computer  system  helps  them  perform  in  their  job  and  type  of  s”stem  used 
are  independent. 


Table  31 

Observed  and  Expected  Frequencies  for  Computer  Type  vs.  the  Perception 
That  Users’  Computer  System  Helps  Them  Perform  in  Their  Job 


MS-DOS  users 

Macintosh  users 

Totals 

Undecided  - 
slightly  rigree 

7 

5 

12 

(6.9) 

(5.1) 

Agree 

85 

43 

128 

(73.58) 

(54.42) 

Strongly  agree 

123 

in 

234 

(134.52) 

(99.48) 

Totals 

215 

159 

374 

Total  Chi-Square  -  6.49 

p  -  .039 

Following  the  procedures  used  throughout  the  study,  the  researcher 
then  conducted  a  one-tail  t-test  to  detect  whether  the  mean  from  one  type 
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of  interface  was  greater  than  the  mean  for  the  other  type  of  interface.  The 
t-test  results  are  reported  in  Table  32  below. 

The  results  of  the  t-test  allowed  the  researcher  to  reject  the  hypothesis 
that  the  means  were  equal.  Though  both  means  were  quite  high  for  a  7- 
point  scale,  the  evidence  presented  in  Table  32  indicates  that  the  mean 
value  for  Macintosh  users  is  statistically  higher  than  that  for  MS-DOS  users. 

Table  32 

T-Test  Results  for  Type  of  Computer  vs.  the  Perception  That  Users’ 
Computer  System  Helps  Them  Perform  in  Their  Job 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

215 

6.535 

.578 

Macintosh  users 

159 

6.654 

.584 

t  value  =  -1.962 

p  =  0253 

From  the  results  of  the  contingency  table  (Table  31 )  and  the  one-tail  t- 
test  (Table  32),  the  researcher  concluded  that  users’  perception  of  the 
extent  to  which  their  computer  helps  them  in  their  job  and  type  of 
computer  used  are  not  independent  variables.  On  the  other  hand,  while  the 
difference  between  graphical  and  text-based  interface  users  is  statistically 
significant,  the  mean  responses  for  each  group  were  extremely  high 
indicating  that  both  sets  of  users  felt  very  strongly  that  their  computer 
helps  them  perform  in  their  job.  The  practical  difference  between  the  two 
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means  is  arguable.  Tests  on  related  questions  in  the  questionnaire  yielded 
no  statistical  difference  between  each  system,  further  indicating  that  both 
sets  of  users  feel  that  their  computer  enhances  their  job  performance. 

Investigative  Question  5 

Investigative  Question  5  examined  whether  there  was  a  difference 
between  types  of  interface  used  and  the  number  of  software  packages  used 
on  the  job.  Because  users  who  used  only  one  software  package  were 
eliminated  under  the  parameters  established  for  this  study,  the  minimum 
number  of  software  packages  used  by  any  respondents  was  two.  Table  33 
presents  the  contingency  table  used  for  this  analysis. 

Table  33 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 

Number  of  Software  Packages  Used  on  the  Job 


Number 

MS-DOS  users 

Macintosh  users 

Totals 

2-3 

68 

32 

100 

(57.82) 

(42.18) 

4-6 

84 

65 

149 

(86.16) 

(62.84) 

More  than  6 

66 

62 

128 

(74.02) 

(53.98) 

Totals 

218 

159 

377 

Total  Chi-Square  -  6.432 

p  -  ,0401 
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The  table  indicates  that  the  number  of  software  packages  used  on  the 
job  and  type  of  computer  system  used  are  not  independent.  Though  the  p- 
value  associated  with  this  test  is  not  as  high  as  seen  in  previous  tests,  it 
nonetheless  falls  within  the  .05  significance  levels  established  for  this 
research.  As  one  can  see  from  the  table,  for  responses  3  and  4,  Macintosh 
users  had  higher  than  expected  frequencies,  while  MS-DOS  users  had  lower 
than  expected  frequencies.  Likewise,  for  response  2.  this  pattern  is 
reversed  with  MS-DOS  users  showing  higher  than  expected  frequencies. 

To  further  examine  whether  the  mean  values  for  MS-DOS  users  was 
actually  lower  than  that  of  Macintosh  users,  the  researcher  again  performed 
a  one-tail  t-test  on  the  data.  Table  34  presents  the  results  of  the  t-test. 

Table  34 

T-Test  Results  for  Type  of  Computer  vs.  Number  of 
Software  Packages  Used  on  the  Job 


The  results  indicated  in  Table  34  suggest  that  users  of  graphical 
interface  systems  use  more  software  packages  on  the  job  than  users  of  text- 
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based  interfaces  (p  =  .0071).  The  results  of  the  t-test,  coupled  with  the 
contingency  table  analysis  presented  in  Table  33.  indicate  that  the  number 
of  software  packages  used  on  the  job  is  not  independent  of  type  of 
computer  system  used.  In  addition,  the  results  of  the  tests  conducted  in  this 
portion  of  the  study  indicate  that  users  of  graphical  user  interfaces  use 
more  types  of  software  packages  on  the  job  than  users  of  text-based 
interfaces  in  the  organizations  studied  in  this  research. 

Investigative  Question  6 

Investigative  Question  6  tested  whether  users  of  one  type  of  interface 
had  higher  levels  of  satisfaction  with  their  computer  system  than  users  of 
the  other  type  of  interface  examined  in  this  study.  9  MS-DOS  users 
strongly  disagreed  that  they  were  extremely  satisfied  with  their  computer 
while  0  Macintosh  users  strongly  disagreed.  7  MS-DOS  users  chose  the 
disagree  option  compared  with  1  Macintosh  user.  Finally,  17  MS-DOS  users 
slightly  disagreed  while  3  Macintosh  users  chose  this  option.  These 
responses  are  not  reported  in  the  table  to  provide  a  more  accurate  picture  of 
the  actual  response  categories  while  still  maintaining  the  required  5 
responses  per  cell.  The  researcher  again  felt  that  including  these  responses 
in  aggregate  form  over  a  wide  response  range  unfairly  distorted  the  actual 
patterns  of  response.  Table  35  presents  the  remaining  observed  and 
expected  values  for  user  satisfaction  with  their  computer  system  and  type 
of  computer  used  on  the  job. 
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Table  35 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
User  Satisfaction  With  Their  Computer  System 


MS-DOS  users 

Macintosh  users 

Totals 

Undecided 

11 

5 

16 

(8.64) 

(7.36) 

Slightly  agree 

49 

12 

61 

(32.94) 

(28.06) 

Agree 

83 

40 

123 

(66.43) 

(56.57) 

Strongly  agree 

39 

98 

137 

(73.99) 

(63.01) 

Totals 

182 

155 

337 

Total  Chi-Square  =  63.378 

p  -  .0001 

The  significance  level  of  the  table  (p  -  .0001)  indicates  that  user 
satisfaction  and  type  of  computer  are  not  independent.  In  examining  values 
in  the  table,  particularly  at  the  high  end  of  the  scale,  one  can  see  that  user 
satisfaction  for  Macintosh  users  was  generally  higher  than  expected  and 
user  satisfaction  for  MS-DOS  users  was  generally  lower  than  expected. 

Since  the  two  variables  were  not  shown  to  be  independent,  the 
researcher  conducted  a  one-tail  t-test  to  determine  if  the  mean  satisfaction 
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rating  for  text-based  interface  users  was  significantly  lower  than  the  mean 
satisfaction  rating  for  graphical  interface  users.  Table  36  below  presents 
the  results  of  the  t-test.  Note  that  all  responses  were  used  in  calculating  the 
t-statistic. 

Table  36 

T-Test  Results  for  Type  of  Computer  vs. 

User  Satisfaction  With  Their  Computer  System 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

215 

5.274 

1.542 

Macintosh  users 

159 

6.396 

.968 

t  value  -  -8.071 

p  =  .0001 

The  results  of  the  t-test  indicate  that  the  mean  satisfaction  level  for 
Macintosh  users  is  higher  than  the  mean  satisfaction  level  foj  MS-DOS  users 
in  the  organizations  included  in  this  study.  The  researcher  was  able  to 
conclude  that  user  satisfaction  with  their  computer  system  was  not 
independent  of  computer  system  used  on  the  job  and  that  graphical 
interface  users  have  a  higher  level  of  satisfaction  with  their  computer 
system  than  text-based  interface  users  in  the  organizations  surveyed  in  this 
study. 
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Investigative  Question  7 

As  Whiteside  has  noted,  it  is  difficult  to  separate  the  interface  from  the 
entire  system  for  purposes  of  evaluation  (27:185).  Investigative  Question  7 
examined  whether  responsible  authorities  rated  the  quality  of  output  higher 
for  one  system  over  the  other  system  considered  in  this  study.  Survey 
respondents  were  asked  if  they  supervised  other  personnel  or  if  they  were 
responsible  for  output  generated  by  a  computer.  Individuals  who 
responded  affirmatively  to  either  question  were  defined  as  responsible 
authorities  for  the  purpose  of  this  research.  To  answer  Investigative 
Question  7,  the  researcher  looked  at  responsible  authorities’  ratings  of 
quality  of  text,  quality  of  graphics,  and  overall  document  appearance  for 
output  generated  by  their  computer  system. 

The  first  area  studied  was  quality  of  text.  Consistent  with  the 
methodology  used  throughout  this  study,  the  researcher  first  looked  at 
expected  and  observed  frequencies  for  quality  of  text  and  type  of  computer 
system  used  to  test  the  two  variables  for  independence.  The  contingency 
table  used  for  this  analysis  is  presented  in  Table  37  on  the  following  page. 
Responses  1-5  were  combined  to  allow  at  least  5  responses  per  cell  in  the 
table. 

The  table  shows  that  Macintosh  users  had  higher  than  expected  ratings 
while  MS-DOS  users  had  lower  ratings  than  expected.  For  example,  107  of 
the  total  150  Macintosh  responsible  authorities  gave  their  system  the 
highest  possible  rating,  while  only  54  of  a  total  187  MS-DOS  users  gave 
their  system  the  maximum  rating.  The  low  p  value  for  the  table  allowed 
the  researcher  to  reject  the  hypothesis  that  responsible  authorities'  ratings 
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of  quality  of  text  and  type  of  computer  system  used  on  the  job  are 
independent. 


Table  37 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Responsible  Authorities’  Ratings  for  Quality  of  Text 


MS-DOS  users 

Macintosh  users 

Totals 

Very  poor  -  above 
average 

67 

8 

75 

(41.62) 

(33.38) 

Good 

66 

35 

101 

(56.04) 

(44.96) 

Very  good 

54 

107 

161 

(89.34) 

(71.66) 

Totals 

187 

150 

337 

Total  Chi-Square  =  70.159 

p  =  .0001 

The  number  of  MS-DOS  users  in  the  lower  categories  was  too  large  to 
eliminate  from  the  analysis;  however,  the  number  of  Macintosh  users  was  so 
small  that  all  of  the  lower  categories  had  to  be  combined  to  meet  the  5 
observed  responses  per  cell  constraint.  Table  38  provides  a  more  detailed 
view  of  the  actual  response  patterns  in  these  lower  categories  and  further 
highlights  the  magnitude  of  the  differe''ces  in  responsible  authorities' 
ratings  for  quality  of  text. 
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Table  38 

Detail  of  Responsible  Authorities'  Observed 
Response  Patterns  for  Quality  of  Text 


MS-DOS  users 

Macintosh  users 

Totals 

Very  poor 

0 

0 

0 

Poor 

1 

0 

1 

Below  average 

11 

0 

11 

Average 

32 

4 

36 

Above  average 

23 

4 

27 
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Based  on  the  results  of  the  contingency  table  analysis,  the  researcher 
conducted  a  one-tail  t-test  to  see  if  the  mean  rating  for  users  of  text-based 
interfaces  was  significantly  lower  than  the  mean  rating  for  users  of 
graphics-based  systems.  Table  39  presents  the  results  of  the  t-test. 

The  t-test  indicates  a  significant  difference  in  the  means  and  suggests 
that  users  of  graphics-based  systems  feel  that  their  system  delivers  better 
quality  output  for  text  than  users  of  text-based  systems. 

Considered  with  the  results  of  the  contingency  table  analysis,  the 
researcher  concluded  that  responsible  authorities’  rating  of  text  quality  is 
not  independent  of  type  of  computer  system  used  on  the  job  and  that  users 
of  graphics-based  system  rate  the  quality  of  text  produced  by  their  system 
higher  than  users  of  command-driven  interfaces  rate  quality  of  text 
produced  by  their  system.. 
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Table  39 

T-Test  Results  for  Type  of  Computer  vs. 
Responsible  Authorities’  Ratings  for  Quality  of  Text 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

187 

5.626 

1.257 

Macintosh  users 

150 

6.633 

.670 

t  value  »  -8.861 

p  =  .0001 

Next,  the  researcher  looked  at  quality  of  graphics  produced  by  each 
type  of  system.  The  methodology  used  for  this  portion  of  Investigative 
Question  7  was  identical  to  that  used  in  examining  quality  of  text  output. 
Table  40  presents  the  contingency  table  used  in  this  analysis.  Again,  a  large 
number  of  categories  in  the  lower  ranges  of  the  response  scale  had  to  be 
combined  for  meaningful  analysis. 

The  pattern  in  Table  40  is  nearly  identical  to  that  in  Table  37.  Again, 
responsible  authorities  using  the  Macintosh  system  rate  the  quality  of 
graphic  output  produced  by  their  system  higher  than  expected  while 
responsible  authorities  using  the  MS-DOS-compatible  systems  had  lower 
ratings  than  were  expected.  As  an  example.  105  of  149  Macintosh  users 
gave  their  system  the  highest  possible  rating.  In  contrast,  only  35  of  182 
MS-DOS  users  gave  their  system  the  highest  rating.  Based  on  this  analysis, 
the  researcher  was  able  to  reject  the  hypothesis  that  responsible  authorities’ 
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ratings  of  output  quality  is  independent  of  type  of  computer  system  used  on 
the  job. 


Table  40 

Observed  and  Expected  Frequencies  for  Computer  '^ype  vs. 
Responsible  Authorities’  Ratings  for  Quality  of  Graphics 


MS-DOS  users 

Macintosh  users 

Totals 

Very  poor  -  above 
average 

84 

8 

92 

(50.59) 

(41.41) 

Good 

63 

36 

99 

(54.44) 

(44.56) 

Very  good 

35 

105 

140 

(76.98) 

(63.02) 

Totals 

182 

149 

331 

Total  Chi-Square  =  102.879 

p  =  .0001 

Again,  the  large  number  of  MS-DOS  users  in  the  lower  ranges  of  the 
scale  was  too  large  to  eliminate  from  the  analysis  while  the  few  number  of 
Macintosh  users  in  the  same  range  forced  the  researcher  to  combine  a  large 
number  of  response  categories.  Table  41  provides  a  more  detailed  look  at 
the  actual  response  patterns  for  these  ranges  and  further  highlights  the 
differences  between  each  of  the  two  groups. 
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Table  41 

Detail  of  Responsible  Authorities’  Observed 
Response  Patterns  for  Quality  of  Graphics 


MS-DOS  users 

Macintosh  users 

Totals 

Very  poor 

1 

0 

1 

Poor 

6 

0 

6 

Below  average 

19 

0 

19 

Average 

39 

3 

42 

Above  average 

19 

5 

24 

Following  the  prescribed  methodology,  the  researcher  next  conducted  a 
one-tail  t-test  on  the  data  to  establish  whether  the  mean  rating  for  one 
interface  was  statistically  different  than  the  mean  rating  for  its  counterpart. 
Table  42  shows  the  results  from  the  t~test  conducted  using  this  data. 

The  results  from  this  t-test  parallel  those  found  for  quality  of  text 
ratings.  The  results  suggest  that  users  of  graphics-based  systems  rate  the 
quality  of  graphics  produced  by  their  system  significantly  higher  than  users 
of  text-based  systems. 

After  analyzing  the  contingency  table  and  t-test  for  responsible 
authorities’  ratings  of  quality  of  graphic  output,  the  researcher  concluded 
that  quality  of  graphics  and  type  of  system  used  on  the  job  are  not 
independent  and  that  responsible  authorities  rate  the  graphics  produced  by 
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graphical  interface  systems  higher  than  the  grapnic  output  produced  by 
text-based  interfaces. 


Table  42 

T-Test  Results  for  Type  of  Computer  vs. 
Responsible  Authorities’  Ratings  for  Quality  of  Graphics 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

182 

5.187 

1.452 

Macintosh  users 

149 

6.631 

.651 

t  value  -  -1 1.247 

p  =  0001 

The  final  area  studied  as  part  of  Investigative  Question  7  was  overall 
document  appearance.  The  contingency  table  used  for  the  initial  analysis  is 
presented  in  Table  43  on  the  following  page.  The  pattern  of  the  results  was 
consistent  with  the  two  previous  contingency  tables  in  this  section  of  the 
study.  Responsible  authorities  who  used  the  Macintosh  system  once  again 
rated  the  overall  document  appearance  higher  than  expected  while  MS-DOS 
users  rated  the  overall  appearance  of  documents  produced  by  their  system 
lower  than  expected.  1 1 1  of  the  total  148  Macintosh  users  gave  their 
system  the  highest  possible  rating,  while  only  35  of  183  MS-DOS  users  gave 
their  system  a  rating  of  7.  Based  on  the  p  value  for  the  contingency  table, 
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the  researcher  rejected  the  hypothesis  that  overall  document  appearance 
and  type  of  computer  system  used  are  independent. 


Table  43 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Responsible  Authorities’  Ratingr  *or  Overall  Document  Appearance 


MS-DOS  users 

Macintosh  users 

Totals 

Very  poor  -  above 
average 

68 

5 

73 

(40.36) 

(32.64) 

Good 

80 

32 

112 

(61.92) 

(50.08) 

Very  good 

35 

111 

146 

(80.72) 

(65.28) 

Totals 

183 

148 

331 

Total  Chi-Square  =  1 12.055 

p  =  .0001 

As  in  the  previous  tables  in  this  section,  the  number  of  MS-DOS  users  in 
the  lower  response  ranges  was  too  large  to  eliminate  from  the  analysis. 
However,  the  few  Macintosh  users  in  these  same  ranges  forced  the 
researcher  to  combine  a  large  number  of  categories.  Table  44  provides  a 
more  accurate  view  of  the  actual  response  patterns  in  the  lower  ranges  of 
the  response  scale. 
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Table  44 

Detail  of  Responsible  Authorities’  Observed 
Response  Patterns  for  Overall  Document  Appearance 


MS-DOS  users 

Macintosh  users 

Totals 

Very  poor 

0 

0 

0 

Poor 

1 

0 

1 

Below  average 

12 

0 

12 

Average 

30 

3 

33 

Above  average 

?5 

2 

27 

The  researcher  then  conducted  a  one-tail  t-test  to  test  for  differences  in  the 
mean  values  for  each  type  of  system.  The  results  of  the  t-test  are  presented 
in  Table  45  below. 

Table  45 

T-Test  Results  for  Type  of  Computer  vs. 

Responsible  Authorities’  Ratings  for  Overall  Document  Appearance 


Group 

Count 

Mean 

Std.  Dev. 

MS-DOS  users 

183 

5.508 

1.195 

Macintosh  users 

148 

6.696 

.602 

t  value  =  -1 1.014 

p  =  0001 
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Consistent  with  the  results  reported  earlier  in  this  section,  the  t-test 
indicated  that  responsible  authorities'  ratings  for  overall  document 
appearance  for  graphics-based  systems  was  significantly  higher  for 
graphics-based  systems  than  text-based  systems  (p  -  .0001).  Based  on  the 
results  of  the  contingency  table  and  the  t-test.  the  researcher  concluded 
that  overall  document  appearance  is  not  independent  of  type  of  computer 
system  used  and  that  responsible  authorities  rate  the  overall  appearance  of 
documents  produced  by  graphical  systems  significantly  higher  than  the 
overall  appearance  cf  documents  produced  by  text-based  systems. 

For  Investigative  Question  7.  each  of  the  areas  studied  (quality  of  text, 
quality  of  graphics,  and  overall  document  appearance)  led  the  researcher  to 
conclude  that  the  quality  of  output  for  graphics-based  systems  is  rated 
much  higher  than  the  quality  of  output  for  text-based  interface  systems  by 
responsible  authorities  in  the  organizations  surveyed  in  this  research.  Each 
statistical  test  yielded  a  significance  level  much  higher  than  could  be 
expected  to  occur  by  chance. 

Investigative  Question  8 

Investigative  Question  8  examined  the  nature  of  the  relationship 
between  users’  job  experience  level  and  type  of  interface  used  on  the  job. 
To  look  at  how  experience  level  affected  type  of  computer  used  on  the  job. 
the  researcher  examined  the  relationships  between  both  the  time  in  the  job 
and  type  of  computer  used,  and  the  respondents'  grade  and  type  of 
computer  used.  Because  the  questions  used  for  this  portion  of  the  study 
were  demographic  in  nature,  the  researcher  limited  his  statistics  to 
contingency  table  analyses  to  test  for  independence. 
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The  first  variable  examined  was  time  in  job.  Table  46  presents  the 
contingency  table  used  for  the  analysis  of  this  variable.  Table  46  shows 
that  respondents  with  less  than  two  years  on  the  job  used  the  Macintosh 
system  more  frequently  than  expected  while  respondents  with  more  than 
three  years  on  the  job  used  MS-DOS-compatible  systems  more  frequently 
than  was  expected. 

Table  46 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 

Respondents’  Time  in  Job 


Years 

MS-DOS  users 

Macintosh  users 

Totals 

Less  than  1 

39 

41 

80 

(46.17) 

(33.83) 

1-2 

35 

44 

79 

(45.59) 

(33.41) 

2-3 

33 

25 

58 

(33.47) 

(24.53) 

3-4 

39 

16 

55 

(31.74) 

(23.26) 

More  than  4 

71 

33 

104 

(60.02) 

(43.98) 

Totals 

217 

159 

376 

Total  Chi-Square  -  17.143 

p  -  .0018 

The  p  value  associated  with  the  table  allowed  the  researcher  to  reject 
the  hypothesis  that  time  in  the  job  and  type  of  computer  system  used  are 
independent.  It  appears  from  the  results  of  the  table  that  users  with  less 
time  on  the  job  are  more  likely  to  use  the  graphics-based  system  while 
users  with  over  three  years  on  the  job  use  the  text-based  interface  in 
greater  numbers. 

The  researcher  then  looked  at  military  status  and  type  of  computer  used 
on  the  job  to  determine  if  there  were  any  potential  dependencies  on 
military  status  as  part  of  the  overall  user  experience  level.  Table  47  shows 
the  observed  and  expected  values  for  military  status  and  type  of  computer 
used  on  the  job. 


Table  47 

Observed  and  Expected  Frequencies  for  Computer  Type  vs. 
Respondents’  Military  Status 


Military  Status 

MS-DOS  users 

Macintosh  users 

Totals 

Military 

51 

78 

129 

(74.16) 

(54.84) 

Civilian 

164 

81 

245 

(140.84) 

(104.16) 

Totals 

215 

159 

374 

Total  Chi-Square  »  25.967 

p  -  .0001 
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The  pattern  ot  results  illustrated  in  Table  47  would  seem  to  indicate 
that  military  users  use  the  Macintosh  more  often  than  expected,  while 
civilian  users  use  MS-DOS-compatible  systems  in  higher  numbers.  This 
marked  difference  may  initially  appear  surprising.  However,  the  results 
from  this  table  may  be  related  to  the  results  from  Table  46  which  examined 
respondents*  time  in  job. 

The  researcher  analyzed  the  data  and  determined  that  time  in  present 
job  is  dependent  on  military  or  civilian  status.  The  mean  time  in  job  for 
military  respondents  was  just  over  one  year,  while  civilian  users  had  a 
mean  time  in  their  present  job  of  well  over  two  years. 

Table  48  presents  the  contingency  table  used  for  this  analysis.  One  can 
see  by  examining  the  table  that  there  were  a  total  of  81  military  users  with 
less  than  2  years  of  experience  while  only  approximately  55  responses  were 
expected  in  these  categories.  On  the  other  hand,  in  the  upper  range  of  the 
response  scale,  there  were  a  total  of  130  civilians  with  at  least  3  years  of 
experience  with  only  slightly  over  103  responses  expected  in  these 
categories. 

Given  these  findings,  the  results  from  Table  47  are  consistent  with  the 
results  from  Table  46,  which  indicated  users  with  less  time  in  their  job  used 
the  Macintosh  in  greater  numbers  than  were  expected.  The  significance 
level  of  the  contingency  table  analysis  (p  -  .0001)  allowed  the  researcher  to 
reject  the  hypothesis  that  grade  and  type  of  computer  system  used  are 
independent. 
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Table  48 

Observed  and  Expected  Frequencies  for  Military  Status  vs. 

Time  in  Job 


Time  in  Job 

Military 

Civilian 

Totals 

Less  than  1  year 

43 

37 

80 

(27.67) 

(52.33) 

1-2  years 

38 

41 

79 

(27.32) 

(51.68) 

2-3  years 

20 

36 

56 

(19.37) 

(36.63) 

3-4  years 

13 

41 

54 

(18.68) 

(35.32) 

4  years  or  more 

15 

89 

104 

(35.97) 

(68.03) 

Totals 

129 

244 

373 

Total  Chi-Square  «  40.723 

p  -  .0001 

Tables  46  and  47  indicate  that  users’  job  experience  level  and  type  of 
computer  system  used  are  not  independent.  The  researcher  concluded  that 
users  with  less  experience  were  more  likely  to  use  graphical  interface 
systems  while  users  with  higher  experience  levels  used  text-based 
interfaces  in  higher  numbers.  In  addition,  military  users  were  more  likely 
to  use  graphics-based  systems  and  civilians  tended  to  use  text-based 
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systems  more  than  was  initially  expected.  However,  readers  must  be 
extremely  cautious  in  interpreting  the  results  presented  for  Investigative 
Question  8  so  as  not  to  assume  dependencies  or  causal  relationships  that 
may  not  actually  exist. 

Summary 

Chapter  4  has  presented  data  analysis  for  this  study.  With  the  exception 
of  Investigative  Question  8,  the  researcher  used  a  combination  of 
contingency  table  analysis  and  t-test  results  in  order  to  present  the  most 
complete  analysis  possible  for  the  data  used  in  this  research.  Investigative 
Question  8  used  only  contingency  tables  to  test  for  independence  due  to  the 
demographic  nature  of  the  variables  examined. 

In  Chapter  4,  each  investigative  question  was  reported  as  a  separate 
entity.  In  reality,  each  separate  investigative  question  forms  only  part  of 
the  overall  picture  for  this  research.  While  the  answers  to  each 
investigative  question  are  significant,  readers  must  not  place  undue 
emphasis  on  any  individual  investigative  question  or  on  one  statistical  test. 
Chapter  V  will  attempt  to  integrate  the  investigative  questions  in  order  to 
attach  meaning  to  the  results  reported  in  Chapter  IV  and  arrive  at  some 
overall  conclusions  and  implications  for  this  research. 
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Chapter  5  presents  the  researcher's  conclusions  and  recommendations 
based  on  the  data  analysis  presented  in  the  previous  chapter.  Chapter  V 
will  review  the  findings  for  each  of  the  investigative  questions  in  an 
attempt  to  integrate  these  findings  to  answer  the  research  question  for  this 
study.  The  chapter  will  then  discuss  potential  implications  from  these 
conclusions  and  their  impact  on  Air  Force  information  resource  managers. 
Finally,  the  chapter  will  present  several  recommendations  for  future 
research. 

Conclusions 

The  research  question  used  for  this  research  was  as  follows: 

What  are  the  advantages  and  disadvantages  of  graphical,  mouse-driven 
user  interfaces  as  compared  to  character-based,  command-driven  user 
interfaces,  and  what  is  each  system’s  impact  on  measures  of  job  efficiency 
and  job  effectiveness  within  the  Air  Force? 

Eight  investigative  questions  were  developed  to  answer  the  research 
question  presented  above.  The  conclusions  reached  for  each  of  the 
investigative  questions  directly  impact  the  overall  conclusions  reached  in 
answering  the  research  question.  Because  of  the  central  importance  of  each 
investigative  question,  it  is  important  to  review  the  conclusions  for  each  of 
the  investigative  questions  here,  before  addressing  the  research  question. 

Investigative  Question  1  studied  how  each  type  of  interface  compared 
in  the  amount  of  time  required  to  learn  the  basic  system.  From  the  data 
gathered  in  the  questionnaire,  the  researcher  concluded  that  users  of 
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graphical  interface  systems  required  significantly  less  time  to  learn  the 
basic  system  than  users  of  text- based  systems.  Consistent  with  this 
conclusion,  there  was  strong  evidence  to  indicate  that  the  amount  of  time 
required  to  learn  the  basic  system  was  dependent  on  the  type  of  computer 
system  (graphics  or  text-based)  used  on  the  job.  Figure  1  indicates  the 
mean  responses  to  the  questionnaire  for  each  type  of  system. 


Figure  1.  Mean  Responses  for  Each  System  for  Time  Required  to  Learn  the 
Basic  System 


Given  that  graphical  user  interfaces  require  less  time  to  learn  the  basic 
system,  the  researcher  then  examined  whether  there  were  differences  in 
the  amount  of  time  required  to  learn  different  applications  programs.  The 
researcher  chose  to  look  at  word  processing,  spreadsheet,  database, 
graphics/presentation,  and  CAD/CAM  applications.  In  addition,  the 
researcher  asked  respondents  to  choose  the  amount  of  time,  on  the  average, 
that  it  took  them  to  learn  a  new  application  program. 
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The  results  for  Investigative  Question  2  were  somewhat  surprising.  For 
word  processing,  spreadsheet,  database,  and  graphics/presentation 
applications,  graphics-based  systems  maintained  a  clear  advantage  in 
learning  time  over  text-based  systems.  Figure  2  presents  the  mean  scores 
for  each  of  these  applications  for  each  type  of  system.  Again,  there  was 
strong  evidence  that  learning  time  for  each  of  these  applications  was 
dependent  on  the  type  of  computer  system  used.  In  each  case,  the 
graphics-based  system  required  significantly  less  time  to  learn  than  text- 
based  systems. 


Figure  2.  Mean  Responses  for  Each  System  for  Time  Required  to  Learn 
Specific  Applications 


However,  there  was  no  evidence  of  such  dependency  for  CAD/CAM 
applications.  One  might  expect  that  text-based  systems  might  maintain  an 
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edge  in  some  of  the  application  areas  studied  here — particularly  those  areas 
(such  CAD/CAM)  where  there  has  been  a  void  (until  recently)  in  the 
number  of  quality  products  for  graphics-based  systems  available.  However, 
there  was  no  evidence  in  this  study  to  support  an  advantage  for  text-based 
systems  in  learning  time  for  any  of  the  application  areas  studied. 

For  the  five  specific  areas  studied,  graphical  systems  offered  a  clear 
advantage  in  four  areas  while  one  area  showed  no  statistical  difference. 
Finally,  when  asked  on  the  average  how  long  it  took  them  to  learn  a  new 
application  program,  users  of  graphics-based  systems  again  required  much 
less  time  than  did  users  of  text-based  systems.  These  results  indicate  that, 
for  most  applications,  graphics-based  systems  offer  significant  advantages 
in  the  amount  of  time  required  to  learn  new  software  packages.  For  other 
application  areas,  there  was  no  apparent  difference  between  text  and 
graphics-based  systems. 

Investigative  Question  3  compared  users’  ratings  of  user-friendliness  for 
each  type  of  system.  To  examine  user-friendliness,  the  researcher  looked  at 
ease  of  use  for  a  list  of  basic  file  operations  and  overall  ease  of  use.  Based 
on  the  results  discovered  for  Investigative  Questions  I  and  2,  it  was  not 
surprising  to  again  find  that  graphical  user  interfaces  were  rated 
significantly  higher  in  users’  ratings  of  user-friendliness.  User-friendliness 
has  long  been  listed  as  one  of  the  primary  advantages  of  graphics-based 
computer  systems  by  the  manufacturers  of  these  systems.  For  the 
organizations  used  in  this  study,  such  claims  appear  to  have  merit.  The  data 
gathered  in  this  research  suggested  that  users’  ratings  of  user-friendliness 
were  dependent  on  the  type  of  computer  system  used  on  the  job.  Figure  3 
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below  illustrates  the  mean  responses  for  each  area  evaluated  for 
Investigative  Question  3. 
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Figure  3.  Mean  Responses  for  Each  System  for  User-Friendliness 


Investigative  Question  4  researched  how  each  type  of  interface  affected 
users'  perceptions  that  their  computer  helps  them  perform  in  their  job.  Just 
because  a  system  is  easy  to  learn,  or  is  rated  higher  in  abstract  concepts 
such  as  user-friendliness,  does  not  necessarily  translate  into  increases  in 
productivity.  However,  if  users  report  that  one  type  of  system  helps  them 
perform  in  their  job  more  than  an  alternative  system,  then  it  is  likely  that 
users  of  that  system  will  be  more  productive  than  users  of  the  other  type  of 
system.  Once  again,  graphical  user  interfaces  offered  a  statistically 
significant  advantage  over  text-based  interfaces.  There  was  evidence  that 
users’  perceptions  of  the  extent  that  their  computer  helps  them  perform  in 


their  job  was  dependent  on  the  type  of  computer  system  they  used  on  the 
job.  Figure  4  illustrates  the  extent  to  which  each  set  of  users  felt  that  their 
computer  system  helps  them  perform  on  the  job.  Though  the  means  for 
each  system  appear  close,  the  difference  was  significant  at  the  .0173  level. 
The  researcher  concluded  that  users’  of  graphics-based  systems  feel  more 
strongly  that  their  computer  helps  them  perform  in  their  job.  However,  in 
real-world  terms  it  is  arguable  whether  a  difference  of  6.654  to  6.535  on  a 
scale  of  7  would  be  noticeable  in  terms  of  higher  productivity  levels.  The 
fact  that  the  means  for  each  group  are  extremely  high  indicates  that  both 
sets  of  users  feel  that  their  computer  plays  a  critical  role  in  helping  them 
perform  on  the  job. 
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Figure  4.  Mean  Responses  for  Each  System  for  the  Amount  Computer  Helps 
Users  Perform  In  Their  Job 

Research  Question  5  studied  another  area  of  potential  productivity  gains 


by  documenting  how  each  type  of  interface  affected  the  number  of 


software  packages  users  of  that  system  used  on  the  job.  Probably  as  a 
result  of  the  training  and  user-friendliness  issues  discussed  above,  graphical 
interfaces  were  statistically  higher  in  this  category  as  well.  Consistent  with 
earlier  results,  there  was  evidence  to  suggest  that  the  number  of  software 
packages  used  was  dependent  on  the  type  of  computer  system  used  on  the 
job.  In  addition,  the  mean  number  of  software  packages  used  was 
significantly  higher  for  graphical  interface  users  over  users  of  text-based 
systems.  Figure  5  illustrates  the  mean  responses  for  each  type  of  system. 
The  difference  illustrated  was  significant  at  the  .0071  level. 


Figure  5.  Mean  Responses  for  Each  System  for  the  Number  of  Software 
Packages  Used  on  the  Job 


This  result  may  be  related  to  the  training  and  user-friendliness  issues 
examined  in  Investigative  Questions  1-3.  If  users  are  able  to  learn  their 
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system  or  an  application  faster,  it  would  seem  likely  that  they  would  be 
more  willing  and  able  to  learn  a  larger  number  of  software  packages.  Also, 
systems  which  are  perceived  as  user-friendly  may  be  less  intimidating  to 
use,  leading  to  a  higher  number  of  software  packages  used  on  the  job. 

Again,  one  must  be  cautious  in  necessarily  attributing  productivity  gains 
to  the  number  of  software  packages  used  on  the  job.  While  certainly 
possible,  productivity  increases  as  a  result  of  the  differences  in  number  of 
software  packages  used  is  likely  dependent  on  what  particular  software 
packages  are  being  used  at  the  margins.  For  example,  if  Macintosh  users 
use  an  additional  graphics  package,  or  filing  system  that  MS-DOS  users  do 
not  use,  then  it  may  be  reasonable  to  assume  that  gains  in  productivity  will 
result.  On  the  other  hand,  if  the  additional  software  packages  are  clock 
programs,  screen  savers,  etc.,  then  attributing  gains  in  productivity  to  the 
use  of  these  programs  is  risky  at  best.  Given  the  difference  in  the  number 
of  software  packages  used,  the  types  of  programs  used  by  each  group  of 
users,  particularly  at  the  margins,  may  be  an  area  for  further  study. 

Investigative  Question  6  examined  user  satisfaction  levels  with  their 
computer  system.  If  users  are  satisfied  with  the  system  they  use,  one  might 
expect  that  the  productivity  would  be  greater  than  for  users  of  a  system 
perceived  as  unsatisfying.  A  system  which  is  satisfying  to  use  reinforces 
using  that  system  to  automate  as  many  appropriate  functions  as  possible 
(remembering  that  it  may  not  be  appropriate  to  automate  some  functions). 
Based  on  the  results  reported  earlier,  it  was  not  surprising  that  graphics- 
based  systems  have  much  higher  levels  of  user  satisfaction  in  the 
organizations  used  in  this  research.  Figure  6  presents  the  mean  responses 
for  user  satisfaction  with  their  computer  system.  It  appears  that  the  level  of 
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satisfaction  is  dependent  on  the  type  of  computer  system  used,  with  users  of 
graphics-based  systems  reporting  much  higher  levels  of  satisfaction  with 
their  system. 
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Figure  6.  Mean  Responses  for  Each  System  for  User  Satisfaction  Level 

Investigative  Question  7  asked  responsible  authorities  to  rate  the  quality 
of  output  generated  by  the  system  they  used  or  were  responsible  for.  To 
address  this  issue,  the  researcher  examined  quality  of  text,  quality  of 
graphics,  and  overall  document  appearance  as  measures  of  output  quality. 

In  each  of  the  three  cases,  graphics-based  systems  had  significantly  higher 
ratings  than  did  text-based  systems.  There  was  strong  evidence  to  indicate 
dependency  between  each  of  the  three  variables  and  type  of  computer 
system  used.  In  addition,  the  mean  scores  for  each  area  illustrated  a  clear 
advantage  for  the  graphics-based  system.  Figure  7  illustrates  the  results 
for  each  category  examined.  The  researcher  concluded  that  responsible 


authorities  in  the  organizations  surveyed  rate  graphics-based  systems  much 
higher  for  output  quality  than  text-based  systems. 


Figure  7.  Mean  Responses  for  Responsible  Authorities’  Ratings  of  Quality  of 
Output 


Primarily  due  to  the  training  and  user-friendliness  findings, 
Investigative  Question  8  used  some  of  the  user  demographics  to  determine 
the  nature  of  the  relationship  between  user  experience  level  and  the  type  of 
interface  used  on  the  job.  Specifically,  the  researcher  checked  to  see  how 
long  a  user  had  been  in  his  job  affected  the  type  of  system  used  on  the  job. 
In  addition,  the  researcher  checked  to  see  how  respondents’  military  status 
affected  the  type  of  system  used  on  the  job. 
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There  was  evidence  to  indicate  that  the  amount  of  time  in  the  job  and 
type  of  interface  used  were  dependent.  Users  who  had  been  in  their  job  for 
less  than  two  years  tended  to  use  the  graphical  user  interface.  On  the  other 
hand,  users  who  had  been  in  their  job  over  three  years  tended  to  use  the 
more  traditional  command-driven  interface. 

Examining  the  data  for  military  status  and  type  of  system  used 
produced  the  most  surprising  results  of  the  study,  in  the  researcher’s 
opinion.  There  was  strong  evidence  to  indicate  that  military  users  used  a 
disproportionate  number  of  graphics- based  systems  while  civilian  users 
used  a  higher  number  of  text-based  interfaces.  Given  the  large  number  of 
text-based  systems  currently  within  use  in  the  Air  Force,  this  result  was  not 
expected  and  has  important  implications  for  information  resource  managers, 
particularly  within  the  Air  Force. 

The  researcher  concluded  that  users'  experience  level  does  impact  the 
type  of  interface  used  on  the  job.  Users  with  less  job  experience  tend  to  use 
graphics-based  systems  while  users  with  more  job  experience  used  text- 
based  systems  in  higher  numbers.  In  addition,  evidence  gathered  from  this 
study  supported  the  hypothesis  that  the  military  status  of  survey 
respondents  and  type  of  computer  system  used  on  the  job  were  not 
independent. 

Each  of  these  independent  questions  addresses  a  separate  facet  of  the 
larger  research  question.  To  answer  the  research  question,  the  researcher 
used  the  inputs  from  the  separate  investigative  questions  to  form  a  macro 
view  of  the  entire  problem  domain.  The  researcher  then  conducted  a 
subjective  analysis  to  arrive  at  the  overall  conclusions  for  the  research 
question. 
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The  consistent  pattern  of  results  for  each  investigative  question  cannot 
be  ignored.  In  each  case,  graphical  user  interfaces  provide  important 
advantages  which  must  be  considered  by  the  information  resource 
manager.  Among  the  more  tangible  advantages  is  training  time.  Both  for 
learning  the  basic  system,  as  well  as  learning  most  types  of  application 
programs,  graphical  user  interfaces  require  significantly  less  time  to  learn 
than  text-based  systems.  Another  more  tangible  benefit  supported  by  the 
data  analysis  is  that  users  of  graphical  user  interfaces  use  more  software 
packages  on  the  job.  As  stated  earlier,  this  may  be  related  to  training  time 
and  other  more  intangible  measures  of  user  acceptance. 

There  were  also  three  more  intangible  benefits  to  graphical  user 
interfaces.  First,  users  of  graphics-based  systems  give  their  system  much 
higher  ratings  of  user-friendliness  than  text-based  interface  users  give  their 
system.  One  would  expect  that  systems  which  are  more  user-friendly  are 
also  more  reinforcing  to  use  and  less  subject  to  user  resistance.  Second, 
users  of  graphics-based  systems  feel  slightly  more  strongly  that  their 
computer  helps  them  perform  their  jobs.  In  a  related  finding,  users  of 
graphical  interface  systems  are  more  satisfied  with  their  computer  system 
than  their  counterparts.  While  only  perceptions,  these  findings  have 
important  productivity  implications  as  workers  who  are  more  satisfied  with 
their  computer  system  and  feel  that  it  helps  them  in  their  job  are  likely  to 
use  their  computer  system  to  its  maximum  potential.  The  final  intangible 
area  addressed  in  this  study  was  responsible  authorities’  ratings  of  output 
quality.  The  results  of  this  study  strongly  suggest  that  the  appearance  of 
documents  (both  text  and  graphics)  produced  by  graphics-based  systems  is 
superior  to  the  appearance  of  documents  produced  by  text-based  systems. 
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Responsible  authorities’  output  ratings  appear  to  be  dependent  on  the  type 
of  system  used  to  produce  that  document. 

Finally,  there  is  evidence  indicating  that  graphical  user  interfaces  are 
particularly  suited  to  individuals  with  lower  experience  levels  on  the  job, 
typically  those  with  less  than  two  years’  experience  on  the  job.  Most 
surprisingly,  a  disproportionate  number  of  military  users  used  the  graphics- 
based  system.  For  organizations  with  similar  demographics  to  those 
presented  here,  graphical  user  interfaces  may  overcome  some  of  the 
difficulties,  particularly  in  the  training  arena,  often  experienced  by 
information  resource  managers. 

The  results  of  this  study  uncovered  no  disadvantages  to  graphical  user 
interfaces.  Readers  should  not  misinterpret  this  to  mean  that  there  are  no 
disadvantages  to  graphics-based  systems— only  that  there  were  no 
disadvantages  in  the  narrow  set  of  variables  examined  in  this  study.  It  is 
important  to  note  that  this  study  did  not  address  networking  or 
compatibility  concerns.  It  is  entirely  possible  that  text-based  systems  may 
offer  important  advantages  in  these  areas. 

Another  critical  factor  which  was  not  addressed  is  cost.  In  many  cases, 
graphical  user  interfaces  have  a  higher  initial  (purchase)  cost  than  text- 
based  systems.  Obviously,  in  many  cases,  initial  cost  may  be  the  single  most 
important  concern.  In  such  cases,  the  traditional  cost-benefit  analysis  is 
probably  more  relevant  to  the  sponsoring  organization  than  the  results 
presented  here.  However,  training,  support,  and  productivity  costs  may 
favor  graphical  user  interfaces  if  cost  is  viewed  from  a  life-cycle 
perspective. 
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It  is  also  important  to  note  that  this  research  is  based  on  an  intensive 
analysis  of  two  unique  organizations  at  Wright-Patterson  AFB.  While  care 
was  taken  to  insure  that  the  tasks  performed  in  these  organizations  was 
fairly  typical  of  other  Air  Force  organizations,  there  are  always  dangers 
associated  with  over-generalizing  the  results  of  specific  research  to 
populations  with  an  organizational  structure  unlike  those  studied  in  this 
research.  Any  conclusions  drawn  for  organizations  not  included  in  this 
study  must  be  done  carefully. 

Implications 

The  results  of  this  research  contain  important  implications  for  Air  Force 
information  resource  managers,  particularly  in  an  era  of  diminishing 
financial  and  personnel  resources.  The  first  area  is  training.  Other  theses 
at  the  Air  Force  Institute  of  Technology  are  currently  underway  to  examine 
the  status  of  computer  training  and  are  beyond  the  scope  of  this  research. 
However,  the  results  of  this  research  impact  the  training  arena.  As  reported 
earlier,  graphical  user  interfaces  offer  significant  advantages  in  both  the 
amount  of  time  required  to  learn  the  basic  system  and  time  required  to 
learn  new  applications  programs.  It  is  not  uncommon  for  low-cost 
microcomputer  resources  purchased  with  grand  intentions  to  remain 
virtually  unused  because  the  personnel  responsible  for  operating  those 
systems  are  either  unable  or  unwilling  to  learn  the  operating  system  and/or 
software  packages  for  that  microcomputer.  To  compound  this  problem,  the 
Communications  (SC)  and  Information  Management  (IM)  functional 
communities  have  not  been  able  to  provide  an  adequate  level  of  training  to 
overcome  the  natural  resistance  that  is  associated  with  change.  Formal 
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training  programs  are  expensive  to  develop,  operate,  and  maintain.  In  an 
era  of  shrinking  budgets,  most  organizations  simply  do  not  have  the  funds 
available  to  dedicate  to  computer  training.  Graphical  user  interfaces  may 
provide  the  answer  to  these  problems.  Because  they  require  less  time  to 
learn  and  are  easy  to  use,  formal  training  programs  become  less  important. 
The  intuitive  nature  of  graphical  user  interfaces  allow  users  to  essentially 
train  themselves.  The  amount  of  time  spent  in  training  is  dramatically 
reduced,  potentially  providing  a  requisite  increase  in  productivity. 

The  second  important  implication  focuses  on  the  job  experience  and 
grade  demographics  discussed  earlier.  Because  of  the  relatively  large 
degree  of  turnover  in  military  work-centers,  there  is  always  a  significant 
number  of  personnel  with  less  than  two  years  experience  in  the  job.  This 
may  explain  why  military  users  use  a  disproportionate  number  of  graphics- 
based  systems.  Because  today’s  Air  Force  work -center  almost  always 
includes  both  inexperienced  and  military  users,  an  interface  which  is  well- 
suited  to  this  population  would  seem  to  offer  the  potential  for  increased 
productivity.  Again,  the  training  issue  becomes  a  central  concern.  Because 
military  members  turnover  quite  frequently,  formalized  training  classes 
often  lose  their  impact  after  one  or  two  years  as  the  people  who  received 
the  training  rotate  to  new  assignments.  For  relatively  inexperienced  and/or 
military  users,  graphical  user  interfaces  offer  an  environment  which 
supports  rapid,  individualized  learning. 

These  are  just  two  implications  which  Air  Force  information  resource 
managers  must  consider  in  completing  systems  requirements  analysis. 

While  text-based  systems  typically  offer  a  lower  initial  or  purchase  cost, 
information  managers  must  consider  organizational  dynamics  as  well  to 
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determine  if,  in  the  long  run,  teit-based  interface  systems  continue  to 
provide  the  most  cost-effective  solution. 

Suggestions  for  Future  Research 

The  results  reported  in  this  study  should  not  be  interpreted  as  an 
endorsement  for  a  particular  graphical  user  interface  or  manufacturer. 
Rather,  this  research  should  stimulate  discussion  and  further  analysis  of  the 
benefits  and  disadvantages  associated  with  graphical  user  interfaces.  In 
particular  there  are  several  areas  which  may  merit  future  study. 

The  research  presented  here  focused  on  two  unique  organizations  at 
Wright-Patterson  AFB.  Based  on  the  results  of  this  study,  it  would  be 
appropriate  to  test  whether  the  results  obtained  in  this  study  could  be 
replicated  across  a  larger  Air  Force  or  Department  of  Defense  population. 

At  the  time  of  this  research,  there  were  very  few  organizations  in  the 
mainstream  of  the  Air  Force  that  used  both  graphical  and  text-based 
interfaces  in  their  daily  computing  operations.  In  all  likelihood,  as  graphical 
user  interfaces  gain  in  popularity,  a  larger  number  of  organizations  with  an 
even  greater  degree  of  functional  diversity  will  be  available  to  test  the 
results  of  this  study. 

Second,  as  more  graphical  user  interfaces  are  implemented,  it  would  be 
appropriate  to  test  a  number  of  graphical  user  interfaces  and  text-based 
interfaces.  As  stated  in  the  methodology  established  for  this  study,  the 
Apple  Macintosh  system  was  chosen  as  a  representative  for  all  graphical 
interfaces  out  of  necessity.  At  the  time  of  the  study,  there  were  very  few 
applications  programs  available  for  graphics-based  systems  except  the 
Macintosh.  As  systems  such  as  Windows  3.0,  as  well  as  others,  become 
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more  available,  the  amount  of  software  available  will  increase  to  an 
acceptable  level.  Research  is  needed  to  identify  whether  the  results  of  this 
study  can  be  applied  to  all  graphical  user  interfaces,  or  whether  different 
graphical  user  interfaces  perform  differently. 

Finally,  research  into  the  life-cycle  cost  of  graphics-based  systems  as 
compared  to  text-based  systems  would  be  useful.  Too  often,  the  purchase 
price  is  the  only  variable  considered  in  choosing  microcomputer  resources. 
Given  the  benefits  presented  here,  research  is  now  needed  to  determine 
which  type  of  system  provides  the  most  cost-effective  solution  over  the  life- 
cycle  of  each  system. 

Summary 

There  appears  to  be  strong  evidence  that  graphical  user  interfaces  offer 
significant  advantages  to  users  of  such  systems.  Graphical  user  interfaces 
consistently  outperformed  text-based  systems  in  training  time,  user- 
friendliness,  user  satisfaction,  number  of  software  packages  used,  perceived 
job  performance,  and  output  quality.  These  results  have  important 
implications  for  future  Air  Force  microcomputing  requirements. 

The  results  presented  here  suggest  a  serious  need  for  future  study  in 
this  area.  For  many  organizations,  the  benefits  gained  from  adopting  a 
graphical  environment  may  outweigh  the  initial  cost  associated  with  many 
graphics-based  systems.  As  graphical  user  interfaces  increase  in  popularity, 
future  studies  should  attempt  to  determine  whether  there  are  differences 
between  graphical  user  interfaces,  or  whether  the  results  presented  here 
may  be  generalized  to  all  graphics-based  systems. 
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The  question  now  shifts  from  whether  some  graphical  user  interfaces 
offer  advantages  over  command-driven  interfaces  to  whether  the 
advantages  reported  here  are  worth  the  associated  additional  initial  cost. 
Air  Force  information  resource  managers  must  consider  the  benefits 
reported  here  if  their  organization  is  to  continue  to  meet  mission 
requirements  in  an  era  when  constraints  on  budgetary  and  personnel 
resources  are  likely  to  increase. 
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Aooendii :  Human-Comouter  Interface  Survey 


Instructions  for  Completing  the 
Hu  man -Computer  Interface  Survey 

1.  Thank  you  for  taking  the  time  to  complete  this  survey.  Attached  you 
should  find  a  questionnaire,  an  answer  sheet  (AFIT  Form  1 1C),  and  a  return 
envelope. 

2.  Please  answer  all  questions  (with  the  exception  of  the  last  4)  on  the 
computerized  answer  sheet  provided.  Use  only  a  number  two  pencil  in 
filling  in  the  number  of  the  appropriate  response. 

3.  Do  not  fill  in  your  name  or  social  security  number  on  the  answer  sheet. 
Your  answers  will  be  combined  with  others  and  will  not  be  attributed  to 
you  personally. 

4.  Be  sure  to  read  the  directions  for  each  section  carefully.  Depending  on 
the  type  of  computer  you  use.  you  will  be  asked  to  answer  different  sections 
of  questions.  This  is  clearly  explained  in  the  directions  preceding  each 
section  of  the  questionnaire.  When  skipping  over  different  sections,  be  sure 
your  answers  on  the  answer  sheet  match  the  question  you  are  actually 
answering  on  the  questionnaire. 

5.  Once  you  have  completed  the  survey,  place  the  questionnaire  and  your 
answer  sheet  in  the  return  envelope  provided.  It  is  important  that  you  not 
fold  or  staple  your  answer  sheet  when  returning  it.  If  you  have  lost  the 
return  envelope,  please  return  them  through  the  base  distribution  system  to 
the  following  address: 

Capt  Michael  G.  Morris 
AFIT/LSG 

Wright-Patterson  AFB,  OH  45433 

6.  Again,  thank  you  for  taking  a  few  minutes  to  assist  with  this  research. 
Your  response  will  help  us  assess  the  impact  different  human-computer 
interfaces  have  in  the  Air  Force  office  environment. 
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HUMAN-COMPUTER  INTERFACE  SURVEY 


Section  1 :  Please  answer  the  following  demographic  questions  on 
the  answer  sheet  provided. 

1.  With  which  organization  are  you  employed? 

1.  Advanced  Tactical  Fighter  SPO 

2.  Wright  Research  and  Development  Center 

2.  Which  category  best  describes  your  primary  job? 

1.  secretarial 

2.  administrative 

3.  management 

4.  engineering 

5.  finance/cost 

6.  computer  operations 

7.  security 

8.  other 

3.  What  is  your  grade? 

1.  El  -E3 

2.  E4-E6 

3.  E7-E9 

4.  01  -  03 

5.  04-05 

6.  06  or  above 

7.  GS-1  -GS-7 

8.  GS-8  or  above 

4.  What  is  the  total  length  of  time  that  you  been  employed  by  the  federal 
government? 

1.  less  than  1  year 

2.  at  least  1  but  less  than  3  years 

3.  at  least  3  but  less  than  5  years 

4.  at  least  5  but  less  than  8  years 

5.  at  least  8  but  less  than  12  years 

6.  1 2  years  or  more 
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5.  How  long  have  you  been  in  your  present  job? 

1 .  less  than  1  year 

2.  at  least  1  but  less  than  2  years 

3.  at  least  2  but  less  than  3  years 

4.  at  least  3  but  less  than  4  years 

5.  4  years  or  more 

6.  What  is  your  highest  level  of  education? 

1.  high  school  graduate 

2.  some  college 

3.  Associate’s  degree 

4.  Bachelor’s  degree 

5.  Master’s  degree 

6.  Doctoral  degree 

7.  Do  you  use  a  computer  on  the  job? 

1.  yes 

2.  no 

If  you  answered  yes  to  question  7.  please  answer  questions  8  - 
11.  If  you  answered  no.  please  skip  to  question  12. 

8.  What  type  of  computer  do  you  use  most  often  on  the  job? 

1 .  Zenith  2-248  or  other  IBM  compatible 

2.  Apple  Macintosh 

3.  other 

9.  How  long  have  you  used  the  computer  specified  in  question  8? 

1.  less  than  3  months 

2.  at  least  3  months  but  less  than  1  year 

3.  1  year  or  more 

10.  Approximately  how  many  hours  per  week  do  you  use  your  computer  on 
the  job? 

1.  less  than  3 

2.  at  least  3  but  less  than  5 

3.  at  least  5  but  less  than  10 

4.  at  least  10  but  less  than  20 

5.  20  or  more 
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1 1.  How  many  different  software  packages  do  you  use  on  the  job? 

1.  1 

2.  2-3 

3.  4-6 

4.  over  6 

12.  Do  you  own  a  computer  at  home? 

1.  yes 

2.  no 

If  you  answered  yes  to  question  12.  please  answer  questions  13  - 
16.  If  you  answered  no.  please  skip  to  question  17. 

13.  What  type  of  computer  do  you  use  at  your  home? 

1.  Zenith  Z-248  or  other  IBM  compatible 

2.  Apple  Macintosh 

3.  other 

14.  How  long  have  you  used  the  computer  specified  in  question  13? 

1.  less  than  3  months 

2.  at  least  3  months  but  less  than  1  year 

3.  1  year  or  more 

15.  Approximately  how  many  hours  per  week  do  you  use  your  computer  at 
home? 

1.  less  than  3 

2.  at  least  3  but  less  than  5 

3.  at  least  5  but  less  than  10 

4.  at  least  10  but  less  than  20 

5.  20  or  more 

16.  How  many  different  software  packages  do  you  use  at  home? 

1.  1 

2.  2-3 

3.  4-6 

4.  over  6 
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All  respondents  should  resume  answering  with  question  17. 

17.  On  a  scale  of  1  to  5,  with  1  being  the  least  advanced  and  5  being  the 
most  advanced,  how  would  you  rate  your  computer  skills? 

1.  1 

2.  2 

3.  3 

4.  4 

5.  5 

Section  2:  The  following  questions  all  pertain  to  the  computer 
you  use  on  the  job  (i.e.  your  answer  to  question  S).  If  you  do  not 
use  a  computer  on  the  job.  please  return  your  questionnaire  at 
this  time.  If  you  do  use  a  computer  on  the  job.  please  answer  on 
the  answer  sheet  provided.  Please  note  that  'actual  working 
time'  refers  to  the  amount  of  time  spent  learning  a  particular 
program  or  skill. 

Ezanple:  day  1:  2  hoars 

day  2:  3  hours 

Actual  working  time  =  5  hours 

18.  Do  you  know  how  to  copy  files  (applications  or  documents)? 

1.  yes 

2.  no 

19.  Do  you  know  how  to  delete  files  (applications  or  documents)? 

1.  yes 

2.  no 

20.  Do  you  know  how  to  create  subdirectories  (folders)? 

1.  yes 

2.  no 

21.  Do  you  know  how  to  rename  a  file  (document)? 

1.  yes 

2.  no 

22.  Do  you  know  how  to  open  a  particular  file  (document)? 

1.  yes 

2.  no 
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23.  Do  you  know  how  to  list/view  ail  files  (documents)  within  a 
subdirectory  (folder)? 

1.  yes 

2.  no 

24.  Do  you  know  how  to  move  a  file  (document)  from  one  subdirectory 
(folder)  to  another? 

1.  yes 

2.  no 

25.  Do  you  know  how  to  format  (initialize)  a  disk? 

1.  yes 

2.  no 

26.  Do  you  know  how  to  copy  (duplicate)  a  disk? 

1.  yes 

2.  no 

27.  How  much  actual  working  time  did  it  take  you  to  learn  all  operations 
listed  in  questions  18  -  26? 

1.  less  than  1  hour 

2.  at  least  1  but  less  than  3  hours 

3.  at  least  3  but  less  than  5  hours 

4.  at  least  5  but  less  than  7  hours 

5.  at  least  7  but  less  than  9  hours 

6.  9  hours  or  more 

7.  have  not  yet  learned  all  operations  in  questions  18-26 

Section  3:  The  following  questions  ask  about  specific  types  of 
applications  you  may  use  on  the  job.  Please  answer  on  the 
answer  sheet  provided.  Again,  please  note  that  "actual  working 
time”  refers  to  the  amount  of  time  actually  spent  learning  the 
program. 

Example:  day  1:  2  hours 

day  2:  3  hours 

Actual  working  time  «  5  hours 

28.  Do  you  use  word  processing  software  on  the  job? 

1.  yes 

2.  no 
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29.  How  much  actual  working  time  did  it  take  you  to  learn  your  primary 
word  processing  program? 

1.  less  than  4  hours 

2.  at  least  4  but  less  than  8  hours 

3.  at  least  8  but  less  than  12  hours 

4.  at  least  12  but  less  than  16  hours 

5.  at  least  16  but  less  than  20  hours 

6.  at  least  20  but  less  than  24  hours 

7.  24  hours  or  more 

8.  do  not  use  my  computer  for  word  processing 

30.  Do  you  use  a  spreadsheet  program  on  the  job? 

1.  yes 

2.  no 

31.  How  much  actual  working  time  did  it  take  you  to  learn  your  primary 
spreadsheet  program? 

1 .  less  than  4  hours 

2.  at  least  4  but  less  than  8  hours 

3.  at  least  8  but  less  than  12  hours 

4.  at  least  12  but  less  than  16  hours 

5.  at  least  16  but  less  than  20  hours 

6.  at  least  20  but  less  than  24  hours 

7.  24  hours  or  more 

8.  do  not  use  a  spreadsheet  on  my  computer 

32.  Do  you  use  a  database  program  on  the  job? 

1.  yes 

2.  no 

33.  How  much  actual  working  time  did  it  take  you  to  learn  the  database 
program  you  are  using? 

1.  less  than  5  hours 

2.  at  least  5  but  less  than  10  hours 

3.  at  least  10  but  less  than  15  hours 

4.  at  least  15  but  less  than  20  hours 

5.  at  least  20  but  less  than  25  hours 

6.  at  least  25  but  less  than  30  hours 

7.  30  hours  or  more 

8.  do  not  use  a  database  program  on  my  computer 
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34.  Do  you  use  a  graphics/presentation  program  on  the  job? 

1.  yes 

2.  no 

35.  How  much  actual  working  time  did  it  take  you  to  learn  your  graphics/ 
presentation  program? 

1 .  less  than  4  hours 

2.  at  least  4  but  less  than  8  hours 

3.  at  least  8  but  less  than  12  hours 

4.  at  least  12  but  less  than  16  hours 

5.  at  least  16  but  less  than  20  hours 

6.  at  least  20  but  less  than  24  hours 

7.  24  hours  or  more 

8.  do  not  use  a  graphics/presentation  program  on  my  computer 

36.  Do  you  use  computer-aided  design/computer-aided  manufacturing 
(CAD/CAM)  software  on  the  job? 

1.  yes 

2.  no 

37.  How  much  actual  working  time  did  it  take  you  to  learn  the  CAD/CAM 
program  you  are  using? 

1 .  less  than  5  hours 

2.  at  least  5  but  less  than  10  hours 

3.  at  least  10  but  less  than  15  hours 

4.  at  least  15  but  less  than  20  hours 

5.  at  least  20  but  less  than  25  hours 

6.  at  least  25  but  less  than  30  hours 

7.  30  hours  or  more 

8.  do  not  use  a  CAD/CAM  program  on  my  computer 

38.  On  the  average,  how  much  actual  working  time  does  it  take  you  to  learn 
a  new  application  program? 

1 .  less  than  4  hours 

2.  at  least  4  but  less  than  8  hours 

3.  at  least  8  but  less  than  12  hours 

4.  at  least  12  but  less  than  16  hours 

5.  at  ieast  16  but  less  than  20  hours 

6.  at  least  20  but  less  than  24  hours 

7.  24  hours  or  more 
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39.  How  many  software  application  packages  do  you  feel  proficient  with? 

1.  1 

2.  2 

3.  3 

4.  4-5 

5.  6-7 

6.  7-8 

7.  more  than  8 

Section  4:  The  Tollowing  section  asks  you  to  rate  the  user- 
friendliness  of  various  aspects  of  the  computer  system  you  use  on 
the  job  (i.e.  your  answer  to  question  8).  In  rating  these 
attributes,  please  use  the  following  scale  in  answering  the 
questions  on  the  answer  sheet  provided: 


1 

2 

1 

3  4 

1  1 

5  6 

1  1 

7 

extremely 

1 

difficult 

1  1 

moderately  moderate 

1  1 

moderately  easy 

extremely 

difficult 

difficult 

easy 

easy 

If  you  use  a  Zenith  or  other  IBM  compatible  most  often  on  the  job. 
please  answer  questions  40  -  47.  If  you  use  a  Macintosh,  please 
skip  to  question  48.  making  sure  to  leave  answers  40  -  47  blank 
on  your  answer  sheet.  If  you  do  not  use  any  of  the  computers 
listed  above,  please  stop  here  and  return  your  questionnaire  and 
answer  sheet  at  this  time.  All  users  will  resume  answering 
questions  at  question  number  56. 

40.  On  the  average,  how  difficult/easy  is  using  the  commands  within  the 
software  packages  you  use  on  the  job? 

41.  How  difficult/easy  is  using  the  keyboard,  as  compared  with  other  input 
devices  you  have  seen  or  used  for  entering  appropriate  commands? 

42.  How  difficult/easy  is  working  with  the  function  keys  (FI  -  FIO)  for 
specific  operations? 
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1 

1 

2 

1 

3 

1 

4 

1 

5 

1 

6 

7 

1 

1 

extremely 

difficult 

1 

moderately 

1 

moderate 

1 

moderately 

easy 

1 

extremely 

difficult 

difficult 

easy 

easy 

43.  How  difficult/easy  is  recalling  the  meanings  of  all  the  function  keys 
between  different  application  programs? 

44.  How  difficult/easy  is  learning  the  meanings  of  function  keys  for  each 
application  program? 

45.  How  difficult/easy  is  moving  the  cursor  to  different  points  around  the 
screen? 

46.  On  the  average,  how  difficult/easy  is  performing  the  basic  file 
operations  named  in  questions  18  -  26? 

47.  Overall,  how  difficult/easy  is  using  your  computer? 

If  you  use  a  Zenith  or  other  IBM  compatible,  please  skip  to 
question  56  making  sure  to  leave  questions  48  -  55  blank  on  your 
answer  sheet.  If  you  use  a  Macintosh,  please  answer  questions 
48  -  55  using  the  scale  below: 


1 

2 

3 

1 

4 

1 

5  6 

7 

1 

extremely 

difficult 

difficult 

I 

moderately 

difficult 

1 

moderate 

moderately  easy 

easy 

1 

extremely 

easy 

48.  How  difficult/easy  is  working  with  the  icons  on  the  Macintosh  in  the 
software  packages  you  use  on  the  job? 

49.  How  difficult/easy  is  understanding  what  each  icon  represents? 

50.  How  difficult/easy  is  working  with  the  Macintosh  windows? 

5 1 .  How  difficult/easy  is  using  the  mouse  for  selecting  specific  commands? 
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1 


2 


3 


4 


5 


6 


7 


extremely  difricult  moderately  moderate  moderately  easy  extremely 
difficult  difficult  easy  easy 


52.  How  difficult/easy  is  learning  to  use  the  mouse? 

53.  How  difficult/easy  is  moving  the  cursor  to  different  points  around  the 
screen? 

54.  How  difficult/easy  is  performing  the  basic  file  operations  named  in 
questions  18  -  26? 

55.  Overall,  how  difficult/easy  is  using  your  computer? 

Section  5:  All  users  should  complete  this  section.  The  following 
statements  solicit  your  attitudes  about  using  your  computer  on 
the  job.  Please  use  the  following  scale  to  respond  to  the 
statements  in  numbers  56  -  64: 

1  2  3  4  5  6  7 


strongly  disagree  slightly  undecided  slightly  agree  strongly 
disagree  disagree  agree  agree 


56.  My  computer  helps  me  perform  in  my  job. 

57.  Using  my  computer  increases  my  productivity. 

58.  Using  my  computer  has  increased  my  chances  of  getting  a  better  job. 

59.  Using  my  computer  has  increased  my  chances  for  promotion. 

60.  Using  my  computer  has  benefitted  my  organization. 

6 1.  My  computer  has  helped  me  do  my  job  better. 

62.  I  am  extremely  satisfied  with  the  computer  I  use  on  the  job. 
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1 

2 

1 

3 

1 

4  5 

1  1 

6  7 

1  1 

strongly 

1 

disagree 

1 

slightly 

1  1 

undecided  slightly 

1  1 

agree  strongly 

disagree 

disagree 

agree 

agree 

63.  I  would  prefer  to  use  a  keyboard-driven,  text-based  interface  such  as 
MS-DOS. 

64.  I  would  prefer  to  use  a  mouse-driven,  graphical  interface  such  as  that 
used  by  the  Macintosh. 

Section  6:  Please  answer  the  following  questions  on  the  answer 
sheet  provided.  You  may  wish  to  double  check  your  answer  sheet 
to  insure  that  you  begin  with  question  number  65. 

65.  Do  you  supervise  other  personnel? 

1.  yes 

2.  no 


66.  Are  you  responsible  for  output  generated  by  a  computer? 

1.  yes 

2.  no 

If  you  answered  yes  to  either  question  65  or  66.  please  answer 
questions  67  -  69  using  the  scale  below: 


1 

1 

2 

3 

1 

4 

1 

5 

1 

6  7 

1  1 

1 

very 

poor 

1 

below 

1 

average 

1 

above 

1  1 

good  very 

poor 

average 

average 

good 

67.  How  would  you  describe  the 

quality  of  text 

produced  by  your  computer 

system? 

68.  How  would  you  describe  the  quality  of  graphics  produced  by  your 
computer  system? 

69.  How  would  you  describe  the  overall  appearance  of  documents  produced 
by  your  computer  system? 


Section  7:  Please  answer  all  of  the  following  questions  which 
apply  to  software  you  currently  use  on  the  job.  Please  answer 
these  questions  directly  on  this  questionnaire. 

70.  What  word  processing  program  do  you  use? 


71.  What  spreadsheet  program  do  you  use? 


72.  What  database  program  do  you  use? 


73.  What  graphics/presentation  program  do  you  use? 


74.  What  CAB/CAM  package  do  you  use? 


Thank  you  for  completing  this  questionnaire.  Please  return  your 
answer  sheet  and  questionnaire  in  the  envelope  provided.  If  you 
have  lost  the  return  envelope,  please  return  the  materials  to  the 
following  address  through  the  base  distribution  system: 


Capt  Michael  G.  Morris 
AFIT/LSG 

Wright-Patterson  AFB,  OH  45433 


Again,  thank  you  for  your  help. 
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